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1  Introduction 


Today’s  electronic  society  is  seeing  increased  demand  for  data  transfer,  internet 
downloads,  online  applications,  video  sharing,  and  storage.  Banks,  companies,  universi¬ 
ties,  and  governments  require  large  secure  data  centers  connected  to  secure  networks. 

This  continuously  drives  the  development  for  more  powerful  servers  and  computer  sys¬ 
tems.  Today’s  data  centers  are  not  single  computer  systems  but  networked  systems  with 
distributed  processing,  computational  power,  and  memory.  The  systems  are  connected  by 
‘nodes’  which  connect  the  pieces  of  the  ‘computer.’  The  configuration  depends  on  the 
applications,  functionality,  and  user  requirements.  This  demanding  environment  is  evolv¬ 
ing  and  pushing  for  the  development  of  higher  speed  interconnects. 

Copper  interconnects  are  currently  used  extensively  within  the  data  center  environment. 
Fiber  optic  technology  is  used  as  a  connectivity  solution  when  a)  higher  performance  is 
required  and  b)  the  cost  differential  compared  to  a  copper  solution  is  affordable.  Fiber 
optic  technology  offers  several  key  advantages  over  copper  solutions  for  data  transmis¬ 
sion.  Within  today’s  data  center,  there  is  ubiquitous  use  of  multi-mode  fiber  and  optical 
transceivers  for  rack-to-rack  connections.  The  fiber  ports  provide  dedicated  links  with 
reach  up  to  300  meters  on  multi-mode  fiber  and  2  km  on  single-mode  fiber.  The  fiber 
connections  allow  networks  to  connect  to  the  wide  area  networks  (WAN)  or  local  area 
networks  (LAN).  As  improvements  have  occurred  in  fiber  optic  transceiver  performance 
and  cost  has  been  reduced,  there  has  been  further  proliferation  of  fiber  optics  within  the 
data  center. 

Traditionally,  there  has  been  a  debate  on  the  distance-cost  crossover  from  a  copper  to  an 
optic  solution.  As  we  delve  into  the  interconnect  market  and  technology,  there  are  several 
motivations  for  optical  implementation.  To  understand  the  industrial  and  commercial 
aspect,  we  look  into  the  applications,  history,  market  drivers,  and  future  issues  that  are 
being  discussed  today. 

OIDA  held  a  one-day  forum  to  increase  the  understanding  on  this  topic  and  to  look  at 
high  speed  interconnects  in  the  data  center.  The  objective  was  to  solicit  input  on  the 
current  issues  related  to  optical  interconnect  technology  which  included  examining  the 
current  research  objectives  of  industry  and  universities  and  understanding  the  commercial 
objectives  and  maturity  of  today’s  technology. 

Several  organizations  within  the  industry  highlighted  the  technology  issues  facing  copper 
interconnects  for  inter-chip,  intra-chip,  and  board-to-board  communications  at  a  Novem¬ 
ber  2004  OIDA  forum.  A  reasonable  question,  considering  the  economic  factors  in  the 
industry  today,  is:  how  can  optical  interconnects  achieve  greater  penetration  of  the  data 
center  and  server  markets?  This  report  reviews  aspects  of  this  subject,  provides  a  synop¬ 
sis  of  the  information  from  the  meeting,  and  presents  several  conclusions. 
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2  Industry  status  and  issues 

The  computer  and  communications  industries  are  interconnected  today  due  in  large  part 
to  the  Internet,  ubiquitous  use  of  data,  and  transport  of  that  data  between  servers,  data 
centers,  and  access  points.  Today’s  computers  are  complex  systems  based  on  several 
technologies.  The  connection  between  the  memory  and  the  central  processing  unit  (CPU) 
is  not  necessarily  on  the  same  board  or  in  the  same  packaged  integrated  circuit  (IC)  chip. 
The  data  center  is  a  distributed  computer  system  with  multiple  nodes  that  have  to  be 
cormected.  These  nodes  sit  at  different  levels,  whether  they  are  on  the  chip,  in  the  server, 
in  the  rack,  or  at  an  alternative  location.  How  these  connections  are  made  is  highly  de¬ 
pendent  on  the  distance  and  the  signaling  speed. 

The  optical  communications  industry  developed  from  the  need  for  people  to  communi¬ 
cate.  The  telecom  industry  has  been  the  driver  for  implementation  of  optical  technology, 
which  offers  several  advantages  over  copper  wireline  communications.  As  the  computer 
industry  moved  to  higher  clock  frequencies,  smaller  process  nodes,  and  multiple  core 
architectures,  fundamental  signaling  issues  drove  the  need  to  look  at  optics  as  an  alterna¬ 
tive  architecture.  Outside  of  the  IC  chip  and  microprocessor,  the  signal  transport  between 
chips  and  distributed  nodes  in  the  server/computer  system  are  being  reviewed  for  more 
extensive  implementation  of  optical  technology. 

Copper  media,  multi-mode  fiber,  or  single-mode  fiber  is  used  in  today’s  data  center  to 
transport  information  between  racks.  The  current  architecture  of  the  servers  and  data 
centers  is  based  on  copper  interconnect  technology.  As  the  speed  requirement  between 
the  different  nodes  increases,  new  approaches  to  the  problem  need  to  be  developed. 
Copper  interconnect  technology  continues  to  evolve  and  improve  as  the  industry  moves 
forward.  The  crossover  point  between  optics  and  copper  solutions  has  been  debated  and 
price  and  performance  are  regarded  as  key  metrics.  Signal  integrity  and  thermal  perform¬ 
ance  are  additional  metrics  that  cannot  be  ignored. 

The  computer  industry  has  several  organizations  that  look  at  the  future  bottlenecks  that 
can  impact  the  industry.  For  the  semiconductor  chip  makers,  the  International  Technol¬ 
ogy  Roadmap  for  Semiconductors  (ITRS)  has  developed  several  roadmaps  and  high¬ 
lighted  areas  of  concern.  Interconnects  is  a  key  area.  ITRS  has  highlighted  a  “red  brick 
wall’’  that  is  looming  for  integrated  circuit  chip  makers.  Optical  technology  offers  a 
solution,  but  the  implementation  and  architecture  to  overcome  the  red  brick  wall  are  not 
understood.  Relative  to  the  traditional  semiconductor  industry,  the  optical  component 
industry  is  immature.  How  the  two  merge  to  solve  the  problem  remains  to  be  seen.  Does 
the  optical  component  base  need  to  be  absorbed  by  the  larger  electronic  semiconductor 
base  of  companies?  Are  there  alternative  hybrid  implementations  that  can  evolve  in 
conjunction?  Are  they  two  different  segments  that  will  continue  to  stay  apart  or  will  there 
be  a  slow  migration  to  optical  implementation  based  on  alternative  optical  technology 
currently  in  use  today?  In  reality,  copper  interconnects  will  continue  to  be  preferred  until 
the  level  of  maturity  and  implementation  for  short  reach  optical  intercormects  can  meet 
several  of  the  requirements  that  semiconductor  chip  companies  expect  today. 
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In  response  to  the  need  for  optical  solutions  to  the  current  problems,  several  technologies 
are  being  investigated.  For  short  reach  links,  vertical  cavity  surface-emitting  laser 
(VCSEL)  technology  is  ubiquitous  today.  The  parallel  VCSEL  array  transmitter  and 
receiver  (transceiver)  is  used  today  for  server  interlinks.  Several  government  and  industry 
initiatives  have  been  investigating  VCSELs  for  inter-board  connections.  The  alternative 
approach  currently  receiving  attention  is  “silicon  photonics,”  i.e.,  the  integration  of  sili¬ 
con  IC  technology  and  III-V  light  emitters.  Silicon  photonics  is  not  a  new  concept:  it  has 
been  investigated  for  more  than  15  years  and  is  utilized  in  production  today  in  communi¬ 
cations  network  equipment.  The  production  of  a  group  IV  light  emitting  structure  has  not 
yet  been  realized;  hybrid  technology  is  believed  to  be  the  best  route  forward. 

As  we  move  to  the  intra-chip  connection,  the  issues  become  more  complex.  Circuit  board 
manufacturing  and  the  reliance  on  FR-4  and  its  properties  are  a  hindrance  to  new  tech¬ 
nology  implementation  and  economies  of  scale  for  higher  performance  materials.  Optical 
technology  for  circuit  boards  is  developing  along  the  path  that  provides  waveguides 
imbedded  in  the  circuit  board.  The  removal  of  high  speed  electrical  signaling  traces  needs 
to  be  complemented  by  changes  in  the  input/output  (I/O)  architecture  of  the  IC  signaling 
pins  and  circuit  card  connectors. 

In  trying  to  understand  the  direction  of  the  interconnect  market  and  technology  paths, 
there  are  several  dimensions  to  the  problem  that  need  to  be  considered.  Some  of  these  are 
commercial  and  market  driven,  others  are  historical  and  experiential.  The  industrial 
direction,  cost/performance,  economy  and  scale  of  the  silicon  industry  vs.  the  communi¬ 
cations  market  need  to  be  considered. 

Economic  issues  have  impacted  the  traditional  optical  transport  business  and  the  current 
optical  components  industry.  This  has  led  to  a  downturn  in  research  and  investment  in 
optical  component  technology  in  the  U.S.  Globally,  investment  in  optics  is  continuing.  In 
Europe  the  7*  Framework  program  has  established  optical  technology  as  a  focus  area. 
Both  in  Europe  and  Japan,  investment  in  optics  is  now  addressing  broader  market  seg¬ 
ments  and  applications.  As  Japan  continues  to  implement  its  fiber-to-the-home  (FTTH) 
program  and  broadband  requirements,  data  center  transport  and  signaling  are  being  ad¬ 
dressed.  The  U.S.  companies  researching  this  area  need  support  from  the  government  and 
a  U.S.  industry  consensus. 
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3  Background 

The  computer  industry  and  the  optical  communications  industry  have  different  require¬ 
ments.  The  market  is  segmented  into  low  cost,  high  volume  PC/laptop  sales,  low-end 
servers,  mid-range  servers,  and  high-end  servers.  Each  segment  has  different  pricing  and 
economic  requirements.  To  look  at  the  interconnect  market  we  need  to  review  several  of 
the  drivers. 

3. 1  Server  market  overview 

The  server/computer  market  is  a  multibillion  dollar  entity.  The  data  centers  utilize  serv¬ 
ers,  data  storage,  and  communication  networks.  The  total  server  market  is  dominated  by  a 
few  well  know  players.  These  companies  utilize  proprietary  and/or  standard  protocols, 
develop  their  own  chipsets,  or  partner  with  major  chip  manufacturers.  The  inside  of  the 
chassis  is  not  interchangeable.  The  total  estimated  market  for  servers  is  around  $56  bil¬ 
lion.  This  estimate  includes  cabling,  processors,  etc.  The  major  players  involved  are  IBM, 
Sun,  Hewlett-Packard,  Dell,  and  Fujitsu.  Figure  1  shows  the  estimated  share  reported  by 
International  Data  Corporation’s  (IDC)  independent  server  market  tracker. 


Others,  $7,390, 


Hewlett-Packard, 
$14,180,  28% 


Figure  1:  Worldwide  server  revenue  and  market  share  for  2005  (US$M) 

(Source:  IDC  Quarterly  server  tracker) 


The  software  which  runs  on  the  servers  is  broken  down  into  three  principle  operating 
systems:  Microsoft  Server,  Linux,  and  UNIX.  The  market  is  currently  deploying  64-bit 
processors  systems  with  multi-core  designs.  The  industry  has  moved  to  multi-core 
designs  rather  than  continuing  clock  frequency  increases  due  to  thermal  and  power 
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management  issues.  Aceording  to  this  tracker,  blade  shipments  accounted  for  around 
4.6%  of  the  total  market  revenue  in  2005.  The  blade  architecture  was  reported  as  growing 
in  popularity.  Fiber  optic  cabling  is  utilized  in  the  ‘high  end’  servers.  The  main  ongoing 
concerns  for  this  market  include  power  consumption  and  thermal  management.  The 
current  I/O  ports  for  the  servers  run  at  varying  signaling  rates.  These  range  from  33  MHz 
(PCI^)  to  more  than  2  Gb/s  (PCI  Express),  with  multi-drop  busing  used  at  the  lower 
speeds  and  point  to  point  links  at  the  higher  speeds. 

3.2  Silicon  industry  technology  drivers 

The  silicon  industry  is  around  a  $200  billion  market.  The  major  microprocessor  and 
memory  manufacturers  employ  state-of-the-art  fabrication  facilities.  These  large  fabrica¬ 
tion  facilities  are  fully  automated  and  can  manufacture  devices  on  300  mm  diameter 
wafers.  The  fabrication  facility  requirements  depend  on  the  market  segment  being  served. 
The  dynamic  random  access  memory  (DRAM),  synchronous  dynamic  random  access 
memory  (SDRAM),  and  microprocessor  segments  are  driven  by  constant  reduction  of 
device  feature  size  to  enable  better  performance  and  reduce  costs.  Figure  2  highlights  this 
driver  in  linewidth  reduction  and  processing  node. 
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Figure  2:  Production  linewidths  in  silicon  (extended  to  2010) 


The  push  in  the  microprocessor  industry  for  smaller  and  smaller  features  has  driven  it  to 
develop  more  precisely-controlled  fabrication  processes  and  implement  different  materi- 
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als,  i.e.,  low-k  dielectrics  and  copper  conductors.  The  push  for  smaller  feature  size  and 
larger  wafers  has  made  a  dramatic  impact  on  the  fabrication  facility  investment  cost.  To 
remain  competitive,  many  companies  transition  to  larger  wafer  size.  Figure  3  highlights 
the  transition  trend  for  wafer  size  production. 


Year 


Figure  3:  Wafer  size  fabrication  transition  trend 

(Source:  1C  Insights) 


The  constant  process  node  trend  and  wafer  size  change  increases  the  investment  cost  for 
semiconductor  fabrication  facility.  Figure  4,  highlights  the  increased  cost  in  facility 
investment  as  the  feature  size  has  decreased. 
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Year 


Figure  4:  Wafer  facility  cost  trend 

(Source:  1C  Insights) 

The  estimated  cost  of  building  a  state-of-the-art  300  mm  wafer  facility  is  now  close  to  $3 
billion.  The  primary  manufacturer  of  microprocessors  is  Intel.  It  is  the  largest  pure  play 
silicon  fabrication  company  and  leads  in  investment  in  semiconductor  technology. 

The  silicon  industry  has  developed  standard  processes  and  design  tool  sets.  The  wafer 
foundry  model  used  by  several  companies  today  was  developed  from  a  U.S.  government 
sponsored  program,  Metal  Oxide  Semiconductor  Implementation  Service  (MOSIS).  The 
standard  design  sets  and  process  tools  enabled  the  silicon  foundry  model.  There  are 
several  electronic  silicon  foundries,  such  as  TSMC,  Charted  Semiconductor,  and  United 
Microelectronics.  These  companies  offer  standard  processes  and  tool  sets  at  different 
process  nodes.  Typically,  they  are  one  generation  behind  the  latest  node  being  imple¬ 
mented  by  the  vertically  integrated  player  Intel. 

For  companies  with  revenue  streams  of  around  $200  million  or  less  annually,  a  fabless 
model  makes  tremendous  sense  and  is  preferred.  It  reduces  the  investment  cost  for  these 
companies  and  allows  them  to  develop  new  design  and  chip  sets  on  essentially  a  shared 
cost  line. 

3.3  Silicon  industry  companies 

The  major  semiconductor  manufacturers  utilize  in-house  fabrication,  foundries,  or  a 
combination  of  both.  The  major  companies  in  this  field  are  outlined  in  Table  1.  The  table 
also  highlights  the  major  foundry  suppliers.  This  list  is  based  on  2005  revenue  streams. 
The  major  semiconductor  manufacturer  is  Intel,  the  principal  CPU  manufacturer. 
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I  Semiconductor  Companies  | 

Company 

Location 

1  Revenue  2005 

|OI  1 

Outsource 

Notes 

Intel  Corporation 

United  States 

$ 

38,826.00 

$ 

8,664.00 

No 

Texas  Instruments  Incorporated 

United  States 

$ 

13,392.00 

$ 

2,324.00 

No 

AMD 

United  States 

$ 

5,847.60 

$ 

165.50 

Freescale  Semiconductor 

United  States 

$ 

5,843.00 

$ 

563.00 

Yes 

Infineon  Technologies  AG 

Europe 

$ 

10,060.30 

$ 

(340.00) 

No 

2006  income 

STMicroElectronics 

Europe 

$ 

8,882.00 

$ 

266.00 

Yes 

Philips  Semiconductors 

Europe 

$ 

7,452.90 

Yes 

NEC  Electronics  Corporation 

Japan 

$ 

6,583.10 

149.10  1 

No 

Matsushita 

Japan 

I  Silicon  Semiconductor  Foundries  | 

Company 

Location 

1  Revenue  2005 

Outsource 

TSMC 

Taiwan 

$ 

8,103.60 

$ 

2,292.70 

United  Microelectronics  Corporation 

Taiwan 

$ 

3,058.40 

$ 

(477.70) 

Chartered  Semiconductor 

Singapore 

$ 

1,032.70 

$ 

(159.60) 

Table  1:  Top  silicon  semiconductor  companies  by  revenue 


The  semiconductor  industry  is  cyclic  in  nature.  The  industry  is  driven  by  consumer  de¬ 
mand,  new  product  development  and  technology  introductions.  The  optical  interconnect 
requirements  today  are  being  driven  primarily  by  companies  that  are  producing  micro¬ 
processors  and  memory  chip  sets. 

3.4  Communications  industry  overview 

The  communications  industry  has  evolved  over  the  last  30  years  as  a  result  of  significant 
changes  in  politics  and  global  requirements.  The  initial  development  of  communications 
networks  was  monopolized  by  the  telecom  operators.  In  the  United  States,  this  was 
AT&T,  in  the  UK  it  was  British  Telecom,  and  in  Japan,  NTT. 

During  the  early  1980s,  politics  had  a  major  impact  on  the  telecommunications  industry 
in  the  U.S.  and  abroad.  In  1984,  the  U.S.  government  decided  to  stimulate  competition  in 
the  telephone  sector.  AT&T  was  broken  up  and  22  local  Bell  companies  were  formed 
into  seven  independent  regional  bell  operating  companies  (RBOC),  often  called  the 
“Baby  Bells.” 

In  the  1990s,  several  significant  events  led  to  over-investment  in  the  industry  which 
resulted  in  the  telecommunications  “bubble.”  The  investment  drivers  in  the  communica¬ 
tion  networks  came  from  the  Internet  and  wireless  telephone  adoption.  The  lighting  up  of 
the  dark  fiber  and  resultant  introduction  of  dense  wavelength  division  multiplexing  (D- 
WDM)  were  major  changes  that  occurred  with  the  wireline  fiber  optic  network.  During 
the  1990s,  most  of  the  major  equipment  vendors  spun  out  their  optical  component  groups. 
This  changed  the  business  and  research  investment  model  for  the  industry  in  the  United 
States  and  Europe.  In  Japan,  the  vertical  nature  remained  due  to  close  links  between  the 
major  component  suppliers  and  the  system  houses. 
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With  the  advent  of  the  Internet,  communication  networks  changed.  The  traditional  tele¬ 
com  networks  were  based  on  voice  traffic,  with  billing  on  a  per  minute  basis.  As  the 
Internet  expanded  and  proliferated,  it  became  dominated  by  data  traffic.  This  has  filled 
the  major  network  pipes  today.  Looking  forward,  the  telecom  carriers  are  now  entering 
the  video  market  and  offer  voice,  video,  and  internet  services  (triple-play  services).  This 
has  led  recently  to  resurgence  in  demand  for  new  network  components.  It  is  expected  that 
video  services  will  dominate  fiber  optic  network  traffic  in  the  future.  As  high  definition 
television  (HD-TV)  proliferates  and  on-demand  video  services  are  offered  to  consumers, 
more  bandwidth  will  be  consumed.  New  video  download  and  sharing  services,  such  as 
You  Tube,  are  also  filling  the  pipes.  These  new  services  are  expected  to  impact  the  data 
center.  For  video-on-demand  service,  one  potential  solution  to  alleviate  metro  traffic 
buildup  is  local  video  caching. 

Today,  most  family  members  have  a  cell  phone.  Figure  5  shows  the  adoption  rate  of 
wireless  phones  in  the  U.S.  Cell  phone  operators  such  as  Verizon,  AT&T,  Sprint,  and 
T-Mobile  have  each  built  large  cell  networks  and  are  now  beginning  to  offer  wireless 
fidelity  (Wi-Fi)  broadband  access  to  the  consumer  and  business  user.  This  new  service 
provides  a  roaming  internet  protocol  (IP)  network,  with  constant  connectivity. 


U.S.  Wireless  Subscribers 


Figure  5:  U.S.  adoption  of  wireless  handsets  from  1990  to  2004 

(Source:  FCC) 

With  the  breakup  of  the  large  vertical  telecom  companies,  a  horizontal  optical  component 
business  model  now  exists  in  the  United  States.  The  current  data  center  requirements  and 
data  transport  services  have  changed  the  network  structure.  Due  to  its  lower  inherent  cost 
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structure,  Ethernet  has  started  to  beeome  the  network  protoeol  of  ehoiee  even  for  the 
telecom  carriers. 

Sinee  the  “teleeommunications  bubble,”  there  has  been  a  switeh  in  revenue  leadership 
among  storage,  LAN,  and  teleeom.  Storage  laser  sales  outstrip  telecom  laser  sales  today. 
This  is  understandable  due  to  the  proliferation  of  optieal  storage  media  and  optieal  video 
players.  The  optieal  storage  market  for  compaet  disc  (CD),  digital  versatile  disc  (DVD), 
high  definition  DVD  (HD-DVD),  and  Blu-Ray  ,  has  different  drivers  than  the  optical 
communications  market.  The  optical  storage  market  is  served  by  several  ehip  manufae- 
turers  that  provide  both  eommunications  and  storage  devices.  The  storage  market  has  a 
different  set  of  requirements  to  enable  read,  write,  and  erase  functions  on  the  optical  disc. 
The  next  generation  optieal  storage  medium  is  the  Blu-Ray  or  HD-DVD  disc.  The  optieal 
heads  now  utilize  three  laser  components  based  on  the  requirement  for  backward  com¬ 
patibility.  The  shortest  wavelength  used  is  405  nm,  whieh  utilizes  InGaN  material  to 
produee  the  source. 

Meanwhile,  the  focus  for  optical  communications  companies  remains  developing  faster 
and/or  high  power  devices.  The  wavelengths  of  interest  are  centered  on  the  transmission 
windows  of  the  optical  fiber,  i.e.,  850  nm,  1300  nm  and  1550  nm.  These  eompanies  are 
also  engaged  in  developing  faster  I/O  optieal  deviees  for  servers  and  data  centers,  but 
remain  cautious  of  the  cost  requirements  for  server  links. 

3.4.1  Optical  components  companies 

Several  eompanies  serve  both  the  eommunieations  and  storage  markets.  The  companies 
that  are  prineipally  manufacturing  devices  for  eommunieations  networks  and  I/O  ports 
sell  into  the  following  application  segments: 

•  Ethernet 

•  Fibre  Channel 

•  SONET/OTN 

•  InfiniBand 

•  proprietary 

The  current  optical  components  vendors  have  seen  tremendous  priee  erosion  in  their 
respeetive  markets.  With  lower  selling  priees  and  high  manufacturing  costs,  most  compa¬ 
nies  have  suffered  significant  losses  sinee  the  “telecom  bubble”  burst.  Table  2  highlights 
the  operating  income  of  several  of  the  major  eomponent  vendors. 
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Company 

2006  Revenues 

Operating 
Income  (01) 

2004  Revenues 

Operating 
Income  (01) 

Notes 

(Millions) 

(Millions) 

(Millions) 

(Millions) 

AVANEX 

$  162.90 

$  (54.70) 

$  106.90 

$  (124.10) 

Finisar 

$  364.30 

$  (24.90) 

$  185.60 

$  (113.80) 

Agilent 

$  7,181.00 

$  349.00 

Note  1 

Avago  T ech 

Private  Company 

EMCORE 

$  143.50 

$  58.70 

$  93.10 

$  (13.50) 

Note  2 

JDSU 

$  1,204.30 

$  (151.20) 

$  635.90 

$  (115.50) 

Cyoptics 

Private  Company 

Private  Company 

Bookham 

$  231.60 

$  (87.50) 

$  79.80 

$  (67.40) 

LuminentOIC 

$  271.70 

$  (10.70) 

OCR 

$  70.10 

$  1.40 

$  57.10 

$  (1.30) 

Eudyna 

- 

- 

- 

- 

Excelight 

Sigma-Links 

Opnext 

$  151.70 

$  (30.50) 

$  79.40 

$  (80.50) 

Mitsubishi 

$  41,048.30 

$  31,917.80 

$  424.40 

Note  1 :  Agilent  sold  the  Optics  piece  and  it  was  formed  as  Avago  Technologies 

Note  2:  Include  Sale  of  GelCore  and  Electronic  materials 

Table  2:  Review  of  current  active  component  vendors  and  their  financial  results 


The  movement  to  transeeiver  modules  compared  to  the  traditional  “golden  box”  discrete 
component  has  commoditized  the  market.  There  are  several  factors  that  have  improved 
the  operating  income  of  several  of  the  fiber  optic  transceiver  companies,  including  the 
resurgence  in  the  volume  of  sales  to  near-2000  levels.  The  optical  components  companies 
have  also  been  actively  engaged  in  downsizing  and  either  pushing  manufacturing  off¬ 
shore  or  to  outsource  production.  Significant  changes  in  the  company  structures  have 
provided  an  ability  to  improve  gross  margins  and  get  closer  to  breakeven.  Each  company 
should  be  reviewed  based  on  its  core  competency  and  product  portfolio.  Both  Finisar  and 
Opnext  have  shown  evidence  of  profitability  in  several  recent  quarters.  Generally,  the 
sector  is  improving  but  is  still  not  healthy  enough  to  allow  increased  spending  on  R&D,  a 
critical  issue  for  the  next  generation  of  networks  and  component  development. 

3.5  Government  initiatives 

Governments  around  the  world  enable  development  of  new  technology  and  research 
through  nationally  funded  programs.  These  programs  impact  the  competitiveness  and 
economic  prosperity  of  a  country.  The  U.S.  government  has  traditionally  funded  research 
in  areas  that  impact  peoples’  lives  through  several  of  its  agencies.  This  section  highlights 
some  of  the  facts  related  to  research  and  development  funding  and  information  for  the 
super  computer  segment  and  its  impact  on  interconnect  development  in  the  United  States 
and  Japan.  The  principal  reason  to  highlight  these  two  countries’  programs  is  the  contin¬ 
ued  race  to  achieve  the  highest  performance  super  computer.  This  race  spins  off  of  devel¬ 
opment  activities  that  impact  the  mid-range  and  low-end  computer  markets  which  benefit 
the  overall  computer  industry. 
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3.5.1  In  the  United  States 


The  U.S.  government  has  been  the  primary  driver  for  high  performance  computer  devel¬ 
opment  over  the  last  40  years.  The  impetus  for  this  has  been: 

1.  National  security  seeds 

a.  Intelligence 

b.  Conventional  and  nuclear  arms 

c.  Weather  forecasting 

2.  U.S.  competitiveness  vs.  the  rest  of  the  world 

a.  Science  and  industry 

3.  Use  of  high  performance  computing  (HPC)  technology 

a.  Medicine 

b.  Climate  research 

c.  Biology 

d.  Chemistry 

e.  Materials 

f  Basic  computer  simulation  (CS)  tools  research 

This  focus  on  HPC  has  enabled  new  advanced  optical  interconnect  devices.  There  are 
several  government  agencies  that  fund  research,  including  Department  of  Energy  (DOE), 
National  Science  Foundation  (NSF),  National  Security  Agency  (NSA),  National  Aero¬ 
nautics  and  Space  Administration  (NASA),  and  the  Defense  Advanced  Research  Projects 
Agency  (DARPA). 

DARPA  has  provided  a  major  focus  for  the  computer  companies  by  funding  programs 
between  industry,  government,  and  universities.  The  level  of  interest  and  investment  by 
the  U.S.  government  in  optical  interconnect  technology  can  be  understood  from  several 
programs  in  which  DARPA  has  invested.  A  number  of  these  programs  are  highlighted  in 
Table  3. 


DARPA  Project  Name 

Period 

Budget  ($MM) 

Analog  Optical  Signal  Processing 

2002-2005 

37 

Chip  to  Chip  Optical  Interconnects 

2003-2007 

45 

Chip  Scale  WDM 

2002-2005 

40 

Data  in  Optical  Domain-  Network 

2002-2006 

60 

Optical  CDMA 

2003-2007 

45 

Photonic  A/D  Technology 

1998-2001 

40 

Note:  Reference  Science  and  Technology  Trends  No. 20  July  2006 

Table  3:  Summary  of  several  DARPA-funded  programs  related  to  optical  interconnects 

(Source:  Science  and  Technology  Review,  Quarterly  Review  No.  20  -  July  2006) 


The  continued  focus  on  next  generation  and  enabling  technology  by  DARPA  and  other 
government  agencies  has  helped  the  U.S.  economy  and  has  enabled  technology  leader¬ 
ship  for  many  U.S. -based  companies. 
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3.5.2  In  Japan 


The  Japanese  government  has  eontinued  to  invest  heavily  in  optieal  communication 
components.  The  size  of  the  fiber-to-the-home  (FTTH)  market  is  a  clear  indication  of  the 
demand  within  Japan  and  the  government’s  goal  for  a  connected  society.  When  we  look 
at  computer  systems  and  servers  there  has  always  been  a  race  between  the  U.S.  and  Japan 
for  the  “biggest  and  best”  super  computer.  The  race  remains  heated,  with  the  U.S.  and 
Japanese  governments  investing  heavily  in  this  area. 

The  Japanese  optical  components  companies  are  researching  several  key  areas  in  long 
wavelength  communications  for  high  data  rate  applications.  For  example,  companies  are 
currently  investing  in  directly  modulated  40  Gbit  distributed  feedback  (DFB)  lasers. 
Several  key  results  were  presented  at  the  Optical  Fiber  Conference  (OFC)  in  2007.  There 
are  a  number  of  agencies  in  Japan  that  invest  in  new  technology.  A  summary  of  the  pro¬ 
grams  funded  by  the  Japanese  government  (Figure  6)  underscores  the  importance  of 
optical  technology  to  the  country. 


List  of  Optical  Communication  related  projects  supported  by  the  Japanese  Government 


FISCAL  YEAR 


f - 

I  Development  of  Optical  i 
I  Interconnects  i 


Figure  6:  Overview  of  Japanese  government  programs  related  to  optical  interconnects 

(Source:  Science  and  Technology  Quarterly  Review  No.  20  -  July  2006) 
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Within  Japan’s  Advanced  Industrial  Science  and  Technology  (AIST)  group,  development 
of  optical  backplanes  is  a  key  focus  area.  Some  results  from  this  group  are  presented  later 
in  this  report. 

3.6  Protocols  for  the  data  center 

This  section  reviews  some  of  the  different  protocols  within  the  network  and  discusses  the 
development  of  the  different  protocols  and  their  current  direction. 

3.6.1  Network  introduction 

The  current  network  infrastructure  is  still  segmented  by  distance,  application,  and  proto¬ 
col.  The  standards  bodies  define  the  technology  implementations  and  specifications.  Data 
centers,  server  farms,  and  high  performance  computer  centers  exist  to  manage  today’s 
information  society.  These  centers  connect  multiple  servers,  CPUs,  memory,  and  storage 
devices.  Each  segment  of  the  network  relies  on  the  three  key  technologies  of  optical, 
electrical,  or  radio  frequency  (RF).  Which  one  is  implemented  is  dependent  on  the  trans¬ 
port  requirement.  A  simple  view  of  the  communications  network  structure  is  given  in 
Figure  7. 


Figure  7:  Simplistic  view  of  t 


lie  current  network  (Core,  Metro,  Access,  and  Last  Mile) 


Transport  over  the  core,  metro,  and  access  is  by  internet  protocol  (IP).  As  we  move  into 
the  data  centers,  the  network  structure  becomes  further  segmented.  The  topology  of  the 
connected  devices  uses  the  same  basic  network  configurations  of  rings,  switches,  and 
hubs,  but  the  requirements  for  transactions,  computation,  and  storage  are  different. 
Within  the  data  center,  several  different  transport  protocols  co-exist.  These  include 
Ethernet,  Fibre  Channel,  InfiniBand,  IP  and,  proprietary  protocols. 

These  protocols  essentially  compete  with  one  another  for  management  of  the  data  center 
network,  but  have  different  latency  and  CPU  overhead  requirements.  The  data  center  is 
an  amalgam  of  several  types  of  sub-system.  The  data  center  requires  fast  access  memory, 
central  processor  units,  storage  (hard  drives/tape  drives),  etc.  It  is  essentially  a  ‘big’ 
computer  providing  computational  efficiency,  data  access,  and  network  functions.  The 
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data  centers  today  manage  tremendous  amounts  of  information.  They  must  be  reliable, 
provide  redundancy,  and  have  minimum  down  time. 

There  a  several  types  of  functions  the  data  center  can  perform,  dependent  on  the  configu¬ 
ration.  In  a  high-performance  computer  system,  the  center  allocates  resources  and 
manages  itself  to  appear  as  a  single  computer  environment.  The  alternative  is  the  imple¬ 
mentation  of  multiple  applications  across  several  platforms  with  dynamic  allocation. 

The  protocols  implemented  in  the  data  center  network  are  dependent  on  the  link  length, 
latency,  and  physical  requirements.  The  following  table  provides  an  overview  of  some  of 
the  implementation  choices  in  use  today: 


Server  to  Server 

Card  to  Card 

Intra-Card 

Length 

10 -300m 

1-10m 

0.3-  1m 

0.1  -0.3m 

No.  of  lines  per  link 

1 

1-10 

1-10 

1-100's 

No.  lines  per  system 

10's 

10-1000’s 

10-1000’s 

10-1000’s 

Standards 

LAN/SAN 

Ethernet 
Fiber  Channel 
Infiniband 

Design  Specific 
LAN/SAN 

Ethernet 
Fiber  Channel 
Infiniband 

Design  Specific 

Ethernet 

Infiniband 

PCI 

Design  Specific 

Fiber  Optic  Transmission 

Yes 

Yes 

? 

? 

Table  4:  Basic  view  of  the  current  protocol  implementation  with  distance 


There  are  several  different  technologies  available  to  store  data  in  the  data  center.  These 
include  magnetic  tape,  hard  drives,  optical  media,  and  standard  computer  memory.  The 
data  center  manager  has  the  choice  of  using  direct  attached  storage  (DAS),  network 
attached  storage  (NAS),  or  a  storage  area  network  (SAN).  The  type  of  interconnect  used 
for  data  storage  is  discussed  in  a  later  section. 

3.6.2  Ethernet 

Ethernet  is  becoming  the  most  pervasive  network  technology  for  communications.  It  is 
used  in  the  data  center,  local  area  networks,  and  wide  area  networks.  There  is  currently  a 
strong  movement  for  Ethernet  to  become  the  protocol  of  choice  for  telecom  service  pro¬ 
viders  as  they  upgrade  their  network  infrastructure. 

The  Ethernet  protocol  is  defined  in  layer  2  of  the  open  system  interconnect  (OSI)  model. 
Its  functions  are  defined  and  developed  by  the  Institute  of  Electrical  and  Electronic  Engi¬ 
neers  (IEEE).  The  original  Ethernet  LAN  architecture  has  evolved  since  its  inception  in 
1973.  The  original  Ethernet  network  used  a  half  duplex  technology  with  carrier  sensed 
multiple  access/collision  detection  (CSMA/CD).  As  it  developed,  it  moved  to  duplex 
operation  and  enabled  the  removal  of  collision  detection,  which  can  cause  network  slow¬ 
down.  During  the  1990s,  the  move  to  a  switched-based  architecture  expanded  the  avail¬ 
ability  of  Ethernet.  A  basic  example  is  shown  in  Figure  8. 
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Figure  8:  Basics  of  the  bridge/switched  base  Ethernet  network  (1994) 


The  level  2  of  the  OSI  model  is  expanded  within  Ethernet  at  the  data  link  and  physieal 
layers.  The  architeeture  in  the  data  link  layer  at  level  2  is  subdivided  in  the  logical  link 
control  (LLC)  and  the  media  access  control  (MAC).  This  is  illustrated  in  Figure  9. 
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Figure  9:  IEEE  802.3  reference  model  up  to  10  Gbit/s  data  rates 

(Source  IEEE  802.3  Standards) 
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The  LLC  defines  the  eommon  interfaee  to  the  network  layer,  i.e.,  it  provides  interface 
points  called  service  access  points  (SAP).  The  MAC  layer  and  the  physical  layer  imple¬ 
ment  the  media  specific  functions.  For  example,  the  MAC  has  direct  communication  with 
a  computer  network  interface  card. 

Today,  Ethernet  has  expanded  to  the  service  provider  backbone  by  using  “carrier  grade” 
Ethernet.  The  Metro  Ethernet  Forum  (MEF),  initiated  in  June  2001,  has  enabled  the 
movement  of  Ethernet  to  the  provider  network.  The  Forum  has  worked  on  several  topics 
to  enable  deployment.  Changes  were  required  as  the  carriers  like  determinism  while  the 
enterprise  customer  likes  plug  and  play.  To  enable  this,  several  topics  on  scalability  have 
been,  and  continue  to  be,  addressed.  A  carrier  operator  is  typically  looking  for  the  follow¬ 
ing  items: 

•  Open  and  standard  interface 

•  Simplified  network  architecture 

•  Scalability 

•  Reliability 

•  Enhanced  charging  functions 

•  Enhanced  security 

•  Quality  of  service  (QoS) 

The  early  Ethernet  network  was  a  best  effort  delivery  system.  The  Ethernet  has  evolved 
to  now  provide  ways  to  manage  information  differently  for  different  user  classes.  The 
MEF  defined  specifications  for  Ethernet  so  that  it  can  act  like  a  service.  The  MEF  certi¬ 
fies  the  equipment  produced  by  its  partners  and  members  which  enables  deployment 
against  a  common  standard.  This  helps  the  carriers  deliver  the  service. 

The  next  generation  Ethernet  network  requires  a  provider  backbone  bridge  as  defined  by 
802.1  ah.  In  this  specification,  each  802.1  ah  level  encapsulates  frames  with  a  new  MAC 
address  and  a  new  service  tag.  The  nesting  level  summarizes  the  MAC  addresses  of  the 
lower  level  with  a  backbone  MAC  address.  The  address  summary  allows  a  high  degree  of 
scaling  without  creating  a  MAC  look-up  table  explosion.  The  end  result  is  that  Ethernet 
deployment  in  the  Metro  has  become  a  realistic  and  reliable  service  solution. 

Ethernet  is  also  deployed  extensively  in  the  data  center.  The  main  issue  with  Ethernet  in 
the  server  environment  is  the  CPU  overhead  required  to  manage  the  connections.  This  is 
important  for  some  applications  and  data  center  structures,  but  less  important  for  others. 
To  reduce  the  server  CPU  utilization,  most  Ethernet  network  interface  cards  (NIC)  utilize 
transmission  control  protocol  (TCP)  offload  engines.  Essentially,  the  NIC  card  has  a 
dedicated  processor  and  off  loads  some  of  the  communications  overhead  from  the  server 
itself.  A  concern  in  the  high  performance  cluster  environment  is  latency.  Latency  can  be 
measured  as  the  time  from  when  a  bit  enters  a  switch  to  when  it  leaves  the  switch.  Due  to 
the  nature  of  the  packet  size  and  routing  functions,  Ethernet  has  issues  with  pass  through 
for  high  performance  computing  environments  compared  to  the  other  protocols  that  are 
available. 


100  Gbit  Interconnects  and  Above:  The  Need  for  Speed  -  18 
Optoelectronics  Industry  Development  Association 


Ethernet  is  in  mass  deployment  in  the  enterprise  environment  at  1  Gbit/s  and  increasingly 
at  10  Gbit/s.  The  enterprise  market  is  now  looking  at  the  next  generation  of  routing  re¬ 
quirements.  The  typical  distance  of  interest  for  Ethernet  solutions  is  less  than  2  km  and/or 
10  km.  Service  providers  are  looking  to  Ethernet  as  the  lowest  cost  solution  for  their 
networks.  Ethernet  has  traditionally  increased  its  data  rate  in  increments  of  lOx  the  previ¬ 
ous  data  rate.  The  802.3  high  speed  study  group  is  currently  reviewing  the  next  MAC  rate 
within  the  IEEE  standards  development  organizations. 


3.6.3  Fibre  Channel 

The  Fibre  Channel  (EC)  protocol  started  development  in  1988  with  American  National 
Standards  Institute  (ANSI)  approval  in  1994.  The  protocol  and  standard  was  developed  to 
provide  a  seamless  storage  and  backup  capability.  The  Fibre  charmel  protocol  allows 
servers  and  workstations  to  connect  directly  to  disk  arrays,  tape  drives,  or  storage  media. 
The  protocol  is  a  dedicated  link  approach  to  allow  fast  access  and  redundancy  across  a 
storage  network.  Typically,  the  FC  network  allows  large  data  transfer  with  low  overhead 
latency  switching  and  minimal  interruptions  to  data  flow.  It  is  different  from  Ethernet  as 
it  uses  credit  flow  for  point  to  point  links.  The  basic  idea  is  that  the  FC  unit  connects  to 
another  unit  and  logs  on.  The  links  then  provide  a  maximum  flow  that  can  be  transmitted 
on  the  dedicated  link.  Communication  occurs  until  the  flow  is  full.  The  FC  network  can 
be  connected  directly  to  the  LAN  or,  today,  even  a  WAN.  This  allows  users  a  seamless 
gateway  to  their  networked  attached  storage  devices  or  servers.  Figure  10  provides  an 
overview  of  three  Fibre  Channel  topologies. 


Figure  10:  Several  basic  Fibre  Channel  topologies 


Fibre  Channel  allows  for  redundancy  and  multiple  storage  capability.  To  connect  to 
alternative  devices  or  network,  FC  utilizes  translation  devices.  The  protocol  allows 
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encapsulation  of  network  and  device  protocols  for  transmission  across  the  FC  fabric.  An 
overview  of  the  translation  devices  is  shown  in  Figure  1 1 . 


Figure  11:  Fibre  Channel  protocol  stack  and  overview  of  the  translation 


The  host  bus  adaptor  (HBA)  is  typically  used  as  the  node  to  connect  to  a  server  or  storage 
device.  These  typically  employ  multi-mode  fiber  optic  transceivers  to  communicate  to  the 
switch  or  port.  The  development  of  Fibre  Channel  is  usually  at  2x  the  previous  data  rate. 
During  the  optical  bubble,  a  10  G  Fibre  Channel  standard  was  developed.  This  has  seen 
minimal  implementation.  Instead  the  industry  stepped  back  and  developed  the  4xFC 
standard.  Currently,  the  T1 1  group  is  working  on  the  8.5  Gbit/s  FC  standard  based  on  the 
10  G  FC  links. 

3.6.4  InfiniBand 

With  the  movement  in  the  data  center  to  more  of  a  cluster  environment,  low  latency  fast 
interconnects  are  required.  The  objective  is  simply  to  allow  more  efficient  use  of  the 
resources  in  the  data  center.  Cluster  computing  requires  data  to  move  at  a  rate  between 
computers  similar  to  the  rate  on  the  board.  InfiniBand  was  initially  developed  to  provide 
a  server-to-server  connection.  It  focused  on  employing  fast  connection  speeds  and  deliv¬ 
ering  the  lowest  possible  latency  (ranging  from  1.8  usee  to  8  usee).  The  additional  re¬ 
quirement  for  InfiniBand  was  to  reduce  the  performance  overhead  placed  on  the  server 
by  managing  the  connection. 

Several  of  the  major  companies  in  the  server  space  are  supporters  of  the  InfiniBand  stan¬ 
dard.  The  principle  benefits  that  it  is  advertised  as  providing  are: 


100  Gbit  Interconnects  and  Above;  The  Need  for  Speed  -  20 
Optoelectronics  Industry  Development  Association 


•  Increased  bandwidth  fabric  interconnects 

•  Low  latency 

•  Enhanced  density  and  reduced  cabling  requirements 

•  Multiple  levels  of  redundancy 

InfiniBand  is  still  a  switched  fabric  network  and  allows  a  point  to  point  solution,  similar 
to  Ethernet  and  Fibre  Channel.  The  architecture  uses  a  single  or  multi-lane  approach  for 
the  transmission  and  receiving  of  data.  Some  of  the  defined  options  for  aggregate  rate  and 
wavelength  are  illustrated  in  Table  5. 


InfiniBand  Fiber 

Attachment  Options 

Attachment 

Option 

Data  Rate 
(Gb/s) 

Wavelength  (nm) 

Fiber  type 

Range 

(Meters) 

IB-1X-SX 

2.5 

850 

50/1 25um 
62.5/125um 

2-250 

2-125 

IB-1X-LX 

2.5 

1310 

SM 

2-10,000 

IB-4X-SX 

10 

850 

50/1 25um 
62.5/125um 

2-125 

2-75 

IB-12X-SX 

30 

850 

50/1 25um 
62.5/125um 

2-125 

2-75 

Table  5:  Examples  of  the  aggregate  rate,  wavelength  and  fiber  options 


It  is  believed  that  more  fiber  links  will  be  required  for  server  I/O  interconnects  as  data 
rates  increase.  With  the  low  latency  achieved  by  Infmiband,  there  has  been  recent  empha¬ 
sis  on  the  advantages  of  InfiniBand  storage  for  blade  server  systems.  Whether  this  will 
increase  market  share  for  InfiniBand  will  depend  on  the  individual  companies  managing 
their  data  center. 

3.7  Standards  development 

Standards  play  a  very  important  role  in  the  development  of  communication  networks  and 
components.  There  are  several  different  standards  bodies  for  the  different  levels  of  net¬ 
work  architecture,  including: 

•  OIF  -  Optical  Internetworking  Forum 

•  IEEE  -  Institute  of  Electrical  and  Electronic  Engineers 

•  IETF  -  Internet  Engineering  Task  Force 

•  ITU  -  International  Telecommunications  Union 

•  TIA  -  Telecom  Industry  Association 

The  use  of  standards,  different  protocol  requirements,  and  technical  specifications  allows 
open  network  configuration  and  compatibility.  Each  organization  has  a  vested  interest  in 
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its  own  requirements  and  development.  The  ITU  is  trying  to  understand  the  next  genera¬ 
tion  telecommunications  network.  IEEE  develops  standard  for  local  area  networks  and  is 
currently  addressing  100  Gbit  Ethernet  as  the  next  generation  data  rate  for  switched 
networks. 

Today’s  data  center  and  servers  utilize  several  of  the  standards  developed  by  these  bod¬ 
ies.  How  the  implementation  of  protocols  changes  within  the  data  center  over  the  next 
decade  is  not  clear.  As  we  look  forward  to  board-to-board  and  intra-chip  optical  intercon¬ 
nects,  different  standards  bodies  with  less  experience  in  optical  technology  may  be  in¬ 
volved. 
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4  Electrical  interconnects  overview 


Within  the  data  center,  there  are  several  different  types  of  interconnects.  To  enable  some 
distinction  between  the  different  requirements  for  optical  technology,  this  report  segre¬ 
gates  the  connections  into  intra-chip,  inter-chip,  and  backplane  interconnects. 

Today,  the  majority  of  connections  are  copper  based.  Optical  solutions  can  enable  higher 
performance  and  fast  data  rates.  The  penalty  that  will  be  paid  as  signals  move  around 
optically  is  associated  with  the  electrical  to  optical  conversion  either  at  the  transmitter  or 
the  receiver.  The  extent  of  this  penalty  is  connection-dependent.  It  needs  to  be  analyzed 
as  part  of  any  cost  or  improvement  driver. 

4. 1  Overview  of  the  interconnect  segmentation 

The  next  few  sections  provide  a  basic  overview  of  the  types  of  interconnects  and  chal¬ 
lenges  associated  with  each. 

4.1.1  Intra-chip  interconnects 

Within  the  IC,  the  individual  transistor  and  circuit  blocks  must  communicate  via  metal 
wires.  As  the  photolithography  line  width  decreases,  the  transistors  become  smaller  and 
faster.  This  leads  to  more  transistors  per  unit  area  and  denser  intra-chip  interconnects. 

The  IC  wiring  hierarchy  typically  uses  shorter  wires  nearer  to  the  silicon  surface  and 
increasing  longer  wires  at  higher  layers.  The  lower  levels  of  interconnect  are  thin  and  are 
used  in  local  routing.  Intermediate  layers  are  of  medium  thickness  and  used  for  semi- 
global  routing.  Finally,  the  top  layers  are  the  thickest  and  are  used  for  global  routing 
across  the  IC.  The  different  levels  of  interconnect  are  customarily  laid  out  in  orthogonal 
directions  to  minimize  cross-talk  between  adjacent  levels.  Furthermore,  this  convention 
helps  to  simplify  routing  patterns.  Figure  12  provides  an  example  of  this  wiring  hierar¬ 
chy. 
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Figure  12:  Example  of  the  metal  inter-connects  layers  in  an  IC 


As  the  clock  frequency  increases  and  the  processing  node  decreases,  global  interconnects 
which  span  the  chip  exhibit  higher  resistance-capacitance  (RC)  time  constants.  This  leads 
to  increasing  interconnect  delay,  transition  time,  and  cross  talk.  In  the  majority  of  com¬ 
plementary  metal-oxide  semiconductor  (CMOS)  processes  used  today,  the  global  inter¬ 
connects  are  typically  broken  up  into  shorter  links  and  signal  is  regenerated  with 
repeaters.  This  breaks  the  long  wires  into  smaller  short  segments  which  decreases  the 
overall  delay.  The  delay  now  becomes  linear  in  nature  rather  than  quadratic.  The  issue 
with  the  repeater  is  the  delay  introduced  by  the  repeater  itself  and  the  associated  power 
penalty  because  of  this  additional  circuitry.  For  processors  operating  at  Gigahertz  clock 
frequencies,  the  global  wires  must  be  designed  assuming  resistance,  inductance,  and 
capacitance  (RLC),  not  just  resistance  and  capacitance. 

The  two  main  parameters  characterizing  the  electrical  interconnects  are  propagation  delay 
and  the  interconnect  bandwidth  density.  The  delay  can  be  reduced  by  increasing  the 
interconnect  width,  but  this  is  at  the  expense  of  a  smaller  bandwidth  density.  As  the 
global  interconnects  are  delay  limited,  it  is  expected  that  optical  interconnects  should  be 
able  to  provide  advantages  here  over  the  current  approach.  The  current  figure  of  merit  for 
delay  optimized  global  electrical  interconnects  is  ImW/mm,  but  this  is  expected  to  in¬ 
crease  in  the  future.  As  the  process  node  is  lowered,  the  electrical  bandwidth  density  can 
increase.  This  provides  a  moving  target  for  optical  interconnects  to  hit.  For  optical  intra¬ 
chip  interconnects,  the  main  challenges  are: 

•  technology  implementation  (silicon  photonics,  organic  waveguides?) 

•  thermal  and  power  consumption 
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•  architecture  and  design  rule  changes 

•  package  design 

4.1.2  Inter-chip  interconnects 

There  is  a  requirement  on  the  circuit  card  to  transfer  data  more  efficiently  between  the 
CPU  and  off  chip  memory.  Chip  packages  today  connect  to  off  chip  memory  using  ball 
grid  array  packages  that  connect  to  the  circuit  card  traces.  The  interconnection  point  and 
trace  can  provide  loss  of  signal  integrity.  To  implement  an  optical  solution,  we  would 
need  to  change  the  I/O  from  optical  to  electrical,  as  shown  in  Figure  13. 


Figure  13:  CPU  and  memory  module  example  configuration 


This  represents  a  significant  departure  from  current  IC  packaging  and  circuit  design. 
There  are  fundamentally  two  elements  that  would  need  to  change: 

1 .  circuit  card  design  and  materials 

2.  I/O  from  the  IC  package 

The  design  of  this  “OE  extension”  module  that  connects  to  the  silicon  circuit  is  not  cur¬ 
rently  defined  by  any  industry  standard.  There  are  several  technical  publications  that 
show  the  potential  of  the  VC  SEE  array  to  connect  to  a  receiver  array  on  board.  The  desire 
to  provide  the  OE  connection  is  a  continuing  challenge  with  the  current  industry  dynam¬ 
ics. 

The  main  implementation  in  practice  today  is  rack-to-rack  connections  with  parallel 
optical  modules  based  on  SNAP- 12  or  POP  4  multi-source  agreements. 


4.1.3  Backplane  interconnects 

Today’s  servers  are  mainframes,  racks,  or  blade  server  configurations.  Copper  back¬ 
planes  connect  between  server  blades  or  cards.  The  copper  connectors  in  the  backplane 
are  produced  by  several  key  manufacturers,  such  as  Tyco,  Teradyne,  Molex,  and  FCI. 
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These  manufaeturers  have  released  a  roadmap  for  copper  connectors  up  to  40  Gbit/s 
(Table  6). 


1Gb/s  2.5Gb/s 

3.125Gb/s  4.0Gb/s 

6.125Gb/s 

10Gb/s 

40Gb/s 

Teradyne  connectors 

HDM 

VHDM&VHDML-Series 

VHDM-HSD 

GBX 

GBX 

ERNI  Connectors 

ERNet 

ERmet 

ErmetZD 

ERmetZD 

ErmetZeroXT 

Connector 

Tyco  Connectors 

HM-ZD 

Z-Pack  HM-ZD 

MultiGig  RT-1 

Z-Pack  HM-ZD 

MultiGig  RT-2 

MultiGig  RT-3 

Technology 

FCI 

Metral  2000 

AIRMAXVS 

Metral  4000 

AIRMAXVS 

AIRMAX 

Fujitsu 

FCN-261Z00X 

FCN260D 

Winchester 

xcell 

SIP-1000 1  platform 

Molex 

Molex  is  teradyne  licensee 

Published  by  iNEMI 

Table  6:  Copper  technology  roadmap  published  by  International  Electronics  Manufactur¬ 
ing  Initiative  (iNEMI) 

(Source:  www.inemi.org) 


The  backplane  provides  for  board-to-board  interconnects.  The  architecture  is  a  switched 
fabric  architecture.  There  are  principally  two  switch  fabric  architectures,  as  highlighted  in 
Figure  14. 


„  Informative:  _  Informative: 

FORCEiiB  a/b  System  Architecture  N+1  System  Architecture 


Figure  14:  Backplane  switching  architecture 

(Courtesy  ofJ.  Goergan,  ForcelO  Networks  -  OiDA  100  Gb  Ethernet  Forum) 


To  move  to  an  optical  backplane,  optical  waveguides  and  connectors  in  the  printed  circuit 
board  (PCB)  need  to  be  developed.  Implementation  would  need  to  be  standardized  by  the 
IPC-Association  Connecting  Electronics  Industries  or  other  standards  organization.  Sev¬ 
eral  companies  are  developing  optical  waveguide  technology  for  the  PCB  with  great 
success,  but  several  key  challenges  remain,  including: 
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•  architecture 

•  fiber/polymer  or  embedded  waveguides 

•  implementation  approach 

•  connector  type 

Optical  will  eventually  be  required.  The  market  and  application  may  not  necessarily  be  in 
servers  first,  but  could  be  in  other  markets  such  as  cell  phones. 

4.2  RF  considerations  for  backplanes  and  connectors 

The  delivery  of  signals  between  processors,  memory,  or  backplanes  today  remain  the 
principle  domain  of  the  electrical  interconnect.  The  main  issues  for  an  increased  data  rate 
using  electrical  connectors  are  electromagnetic  interference  (EMI)  performance,  noise, 
attenuation,  signal  integrity,  and  cross-talk.  Because  of  these  issues,  the  prospect  of  using 
fiber  optic  solutions  is  more  appealing.  For  electrical  signals,  the  impedance  of  the  path  is 
dependent  on  the  dielectric  material  constants,  mechanical  construction,  and  conductor 
materials.  The  simplest  way  to  think  of  the  connector  is  as  a  transmission  line.  Reflec¬ 
tions  need  to  be  minimized  to  ensure  that  the  signal  remains  “clean.”  This  requires  care¬ 
ful  control  of  all  aspects  of  the  connector  mechanics  and  design.  For  example,  a  plated 
through  hole  (PTH)  is  typically  viewed  as  a  capacitive  lumped  element.  Above  2.5 
Gbit/s,  the  impact  of  the  through  hole  must  be  viewed  as  a  transmission  line  with  a  shunt 
element  (or  stub).  This  complicates  matters  as  the  plated  through  hole  must  be  analyzed 
as  a  microwave  structure. 

To  improve  the  signal  integrity  of  copper  cables  and  connectors,  microwave  design  rules 
and  modeling  must  be  implemented.  The  simplest  approach  is  to  use  microwave  strip 
lines  and  transmission  line  theory.  Reflections  and  losses  must  be  minimized  to  provide 
good  signal  integrity.  There  are  multiple  approaches  for  transmission  lines  on  the  circuit 
board.  Several  useful  transmission  line  examples  are  shown  in  Figure  15. 


Micro  Strip 


Strip  Line 


CPW 


CB-  CPW 


Figure  15:  Different  transmission  line  geometries 
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The  interaction  of  the  transmission  line  is  dependent  on  several  factors.  One  critical 
factor  is  the  dielectric  loss.  Table  7  highlights  different  circuit  card  substrate  materials 
that  can  be  used  at  high  frequencies. 


Material/Process 

Microwave  Printed  Circuit 

Thin  Film 

Thick  Film 

Co-fired  Glass 

Cofired 

Ceramic 

Ceramic 

(LTCC) 

(HTCC) 

Base  Substrate 

PTFE  Glass  Fiber 

AI203 

AI203 

NA 

NA 

PTFE  Ceramic 

AIN 

AIN 

Polyester  Glass 

Hydro  carbon  ceramic 

BeO 

Quartz 

Glass/Ceramic 

BeO 

Conductors 

Cu 

Au,  Al,  Cu 

Au,  PtAu 

Au 

W 

Ag,  PdAg 

Cu 

Ag 

PdAgCu 

Mo 

Dielectrics 

N/A 

Si02 

Glass  Ceramics 

Glass  Ceramic 

Ceramic 

Polyimide 

BCB 

AI203  tape 

Resistors 

N/A 

NiCr 

Ru02 

Ru02 

NA 

TaN 

Doped  Glass 

Doped  Glass 

Table  7:  Examples  of  material  that  can  be  used  at  very  high  frequencies 


For  mass  production  FR-4  material  is  primarily  used  in  today’s  manufacturing  process. 
Other  materials  like  Rogers  4003  provide  an  order  of  magnitude  improvement  in  dielectric 
constant  control  for  RF  applications.  The  cost  of  these  ceramic  materials  has  decreased 
through  widespread  adoption  in  the  wireless  cell  phone  market.  To  ensure  that  the  signal 
integrity  is  maintained,  different  microwave  models  can  simulate  the  signal  traces  and 
interactions.  These  include  analog-to-digital  converter  (ADC),  high-frequency  structural 
simulator  (HFSS),  and  lumped  element  models.  The  scattering  parameters  (S-parameters) 
for  the  connector  are  the  simplest  to  measure  and  model.  This  allows  one  to  understand  the 
reflection  and  transmission  characteristics  of  the  connector/cable  assembly.  As  we  move  to 
very  high  data  rates,  the  connector  and  cable  assembly  need  to  viewed  as  one  system. 

In  the  backplane,  there  are  additional  techniques  that  can  be  employed  to  improve  the 
received  signal.  These  include  pre-emphasis,  multi-level  adaptive  signaling,  or  clock  and 
data  recovery  (CDR).  CDR  uses  an  electronic  IC  to  recover  the  signal  and  retransmit  it  to 
the  next  connection  point.  With  pre-emphasis,  some  of  the  frequencies  of  the  signal  are 
boosted  so  that  the  loss  impairments  transform  the  signal  to  produce  a  clean  signal  at  the 
other  end  of  the  connector.  These  techniques  are  extending  the  ability  of  copper  cables 
and  connector  to  achieve  higher  data  rates  and  transmission  distances. 

4.3  Channel  model 

For  the  backplane,  a  channel  model  helps  provide  specification  requirements  for  manu¬ 
facturers  to  meet.  This  type  of  informative  model  has  been  adopted  by  several  standards 
bodies.  It  allows  parameterization  and  design  criteria  for  the  channel  to  be  defined.  Fig¬ 
ure  16  shows  the  basic  overview  of  the  channel  model  for  a  chassis  connection. 
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Figure  16:  Example  of  a  channel  model 

(Courtesy  ofJ.  Goergan,  ForcelO  Networks  -  OIDA  100  Gb  Ethernet  Forum) 

The  trace  length  can  be  fixed  or  the  maximum  distance  defined  for  the  channel  based  on 
current  circuit  board  material.  An  informative  methodology  can  be  employed  which  uses 
a  fitting  equation  for  signal  loss.  Figure  17  shows  and  example  of  the  SDD21  of  a  chan¬ 
nel  model. 
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Figure  17:  Example  of  a  real  channel  model 

(Courtesy  of  J.  Goergan,  Forcel 0  Networks) 
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4.4  ITRS  roadmap  for  interconnects 


The  International  Technology  Roadmap  for  Semiconductors  (ITRS)  has  defined  a  path 
for  the  next  generation  of  technology  nodes  (Table  8).  As  part  of  the  road  mapping  proc¬ 
ess,  the  issues  facing  the  semiconductor  industry  are  discussed  in  this  report. 


Year  of  Production 

2006 

2007 

2008 

2009 

2010 

2011 

2012 

2013  .... 

2020 

DRAM  Stagger  contacted  metal  1  (M1)  1/2  Pitch  (nm) 

70 

65 

57 

50 

45 

40 

36 

32 

14 

MPU/ASIC  stagger  contacted  metal  1  (Ml)  1/2  Pitch  (nm) 

78 

68 

59 

52 

45 

40 

36 

32 

14 

Flash  Uncontacted  Poly  Si  1/2  Pitch  (nm) 

64 

57 

51 

45 

40 

36 

32 

28 

13 

MPU  Printed  gate  length  (nm) 

48 

42 

38 

34 

30 

27 

24 

21 

9 

MPU  Physical  Gate  Length  (nm) 

28 

25 

23 

20 

18 

16 

14 

13 

6 

jsource:  ITRS  2006  up-date  | 

Table  8:  Predicted  year  of  production  and  process  node  line  width 

(Source:  ITRS  Roadmap) 


Several  bottlenecks  have  been  predicted.  A  number  of  these  issues  were  reported  at  a 
November  2004  OIDA  forum  on  interconnects.  Two  areas  were  highlighted  that  optical 
device  technology  might  be  able  to  address: 

1 .  I/O  pin  count  versus  signaling  rate  per  pin 

2.  intra-chip  interconnects  on  the  intermediate  and  global  level  (“red  brick  wall”) 

These  two  areas  are  being  addressed  by  research  laboratories  to  find  a  potential  solution. 
It  is  believed  that  continued  advances  in  copper  interconnect  technology  will  dominate 
for  the  next  5  to  1 0  years  until  these  bottlenecks  can  no  longer  be  addressed  by  electrical 
solutions. 

4.5  3-D  stacking  and  wafer  ievel  interconnects 

One  current  approach  to  providing  a  faster  connection  to  memory  on  the  chip  is  to  stack 
die  on  top  of  one  another.  This  approach  reduces  the  interconnect  length  and  allows 
higher  speed  interconnects.  Currently,  microprocessors  are  moving  down  the  path  of 
multi-core  architectures.  The  advantage  of  the  multi-core  approach  is  the  management  of 
thermal  load  and  heat  sinking.  As  the  implementation  of  on-chip  optical  interconnects 
remains  a  considerable  challenge,  the  approach  today  is  to  extend  the  useful  life  of  cop¬ 
per  interconnects.  It  is  currently  believed  that  a  resilient  mesh  approach  will  allow  tradi¬ 
tional  electrical  signaling  to  be  used.  As  the  processors  which  require  memory  and  video 
cards  become  more  process-intensive,  the  off-chip  memory  access  will  require  high 
interconnection  speed.  There  are  two  ideas  that  can  be  pursued  to  address  this:  die  co¬ 
packaging  or  die  stacking.  Figure  18  highlights  these  approaches. 
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Integration  Technologies  to  Address 
BW  and  Power  Challenges 

Multi-chip  package  3D  stacking 


BW  (GB/s) 


Die  Flexibility  High 


Figure  18:  Integration  of  memory  methodologies  being  pursued  today 

(Courtesy  of  J.  Bautista,  Intel  Corporation) 


The  stacking  of  die  using  via  technology  for  connection  is  being  developed/implemented 
by  several  companies.  In  this  approach,  a  via  in  a  silicon  spacer  allows  direct  access  to 
the  memory  chip  from  the  CPU.  Figure  19  shows  an  implementation  for  an  80  core  proc¬ 
essor  under  development  at  Intel  Corporation. 


Figure  19:  Polaris  80  core  and  3-D  SRAM 

(Courtesy  of  J.  Bautista,  Intel  Corporation) 
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In  this  example,  80  SRAM  tiles  connect  to  the  underlying  IC  circuit.  This  approach  of  3-D 
stacking  of  die  delays  the  need  to  provide  an  optical  solution  for  off  chip  memory  connection. 
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5  Optical  interconnects  for  backplane  and  boards 

There  are  several  requirements  for  the  optical  interconnect  to  move  further  toward  the 
chip  domain  in  computer  and  server  systems.  The  current  technology  in  use  today  is 
based  on  the  switch  and  router  requirements  for  long  distance  optical  networks.  The 
dichotomy  in  the  area  is  between  short  reach  links  (links  <  300  meters  in  length)  and  long 
reach  links  (>  300  meters).  The  breakdown  in  optical  networks  is  based  on  the  link 
budget  requirements  and  so  has  been  segregated  into  transmitter  and  receiver  type  vs. 
distance  and  bit  rate.  A  classic  example  is  the  SONET  architecture.  Tables  9  and  10  show 
the  classification  imposed  on  links  for  transport  networks. 


Application 

Intra-Office 

Inter-  Office 

Short  Haul 

Long  Haul 

Source  Nominal  Wavelength  nm 

1310 

1310 

1550 

1310 

1550 

Type  of  Fiber 

Rec. 

Rec. 

Rec. 

Rec.  G652 

Rec! 

Rec. 

G652 

G652 

G652 

G652 

Rec. 

G653 

Distance  (Km) 

<2 

15 

40 

80 

STM-1 

1-1 

S-1.1 

S-1.2 

L-1.1 

L-1.2 

L-1.3 

STM  Level 

STM-4 

1-4 

S-4.1 

S-4.2 

L-4.1 

L-4.2 

L-4.3 

STM-16 

1-16 

S-16.1 

S-16.2 

L-16.1 

L-16.2 

L-16.3 

Table  9:  Application  table  from  the  ITU,  specifying  link  length  classification 

(Source:  ITU-T  G957  Specification) 


Unit 

Values 

Digital  Signal 

STM-16  according  to  Recommendations  G707  and  G958 

Nominal  Bit  rate 

kbit/s 

2488320 

Application  code 

1-16 

S-16.1 

S-16.2 

L-16.1 

L-16.2 

L-16.3 

Operating  Wavelength  range 

nm 

1266-1360 

1260-1360 

1430-1580 

1280-1335 

1500-1580 

1500-1580 

Transmitter  at  reference  point  S 

Source  type 

MLM 

SLM 

SLM 

SLM 

SLM 

SLM 

Spectral  Characteristics 

-  Maximum  RMS 

nm 

4 

-Maximum  -20dB  width 

nm 

1 

<1 

1 

<1 

<1 

-minimum  side  mode 

dB 

30 

30 

30 

30 

30 

-suppression  ratio 

Mean  Launched  Power 

-  maximum 

dBm 

-3 

0 

0 

3 

3 

3 

-minimum 

dBm 

-10 

-5 

-5 

-2 

-2 

-2 

Minimum  Extinction  ratio 

dB 

8.2 

8.2 

8.2 

8.2 

8.2 

8.2 

Table  10:  Application  table  specifying  transmitter  and  receiver  requirements 

(Source  ITU-T  G957  Specification) 


With  these  standards  specified  for  networks,  optical  components  have  moved  to  a  mod¬ 
ule-based  component  structure.  The  modules  are  referred  to  as  transceivers  and  contain 
memory,  driver,  and  receiver  chip  sets,  and  can  contain  MUX  and  DE-MUX  functions. 
The  actual  complexity  of  the  module  depends  on  the  form  factor  and  requirements  from 
the  customer  of  the  switch,  router,  or  host  bus  adaptor.  The  next  section  provides  a 
review  of  the  current  types  of  transceivers.  For  server  and  board  interconnects,  the  ques¬ 
tion  on  applicability  may  require  alternative  standard  and  reliability  requirements.  For 
intra-chip  and  inter-chip  connections,  the  actual  “module,”  might  develop  into  different 
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form  factors  than  those  readily  available  today.  The  main  issue  for  the  current  entrenched 
supply  base  is  the  lack  of  incentives  and  investment  money  to  exploit  a  potential  high 
volume  server  chip  interconnect  market. 


5. 1  Fiber  optic  connectors  and  cabie  assembiies 

The  standard  fiber  optic  components  in  servers  and  switches  today  are  transceivers  and 
fiber  optic  cable  assemblies.  These  are  based  on  well  known  and  mass  manufactured 
glass  fiber.  For  circuit  board  routing,  optical  waveguide  is  an  alternative  device  technol¬ 
ogy  that  can  be  employed.  For  Fibre  Channel  SANs  and  Ethernet  LANs,  fiber  optic 
components  utilize  both  single-mode  and  multi-mode  fiber  cable  assemblies.  These  are 
typically  duplex  (two)  fiber  connections.  For  servers  using  proprietary  and/or  InfiniBand 
connections,  parallel  optic  modules  are  utilized.  Parallel  modules  use  ribbon  fiber  cable 
assemblies.  The  different  connectors  for  the  fiber  cable  used  today  are  MT-RJ,  LC,  SC 
and  MPO  connectors.  All  of  these  connectors  provide  a  different  ferrule  size  and  fiber 
spacing.  Table  1 1  provides  dimensions  of  each  connector  type. 


Connector  Type 

Ferrule  Diameter 

Fiber  spacing 

SM/MM 

ST 

2.5mm 

Both 

SC 

2.5mm 

Both 

LC 

1.25mm 

6.25mm 

Both 

MT-RJ 

2.5mm  x  4.5mm 

0.75mm 

Both 

MPO 

3mm  X  7mm 

0.25mm 

MM 

Table  11:  Ferrule  dimension  and  connector  types 


For  high  density  applications,  the  multi-fiber  push  on  (MPO)  fiber  cable  assembly  is  pre¬ 
ferred.  The  cable  assembly  can  be  fanned  out  to  other  connector  types  or  can  simply  be 
MPO-to-MPO.  Alternative  fiber  management  techniques  have  been  developed  where  effec¬ 
tively  the  fiber  is  encased  in  a  plastic  jacket  and  routed  with  terminated  connectors  at  either 
end  of  the  assembly.  This  approach  is  simple  but  custom  to  the  chassis  being  used.  With  the 
fiber  connector  technology,  if  the  transceiver  solution  goes  parallel,  there  are  two  choices: 

1 .  Combine  multiple  wavelengths  down  a  single  fiber  -  e.g.,  D-WDM/CWDM 

2.  Utilize  parallel  arrays,  e.g.,  2-D  and/or  1-D  array  ribbon  fiber 

Fiber  optic  cable  is  used  extensively  to  transmit  data  across  communication  networks.  It 
was  first  developed  for  the  telecom  network  in  the  early  1970s.  Improvements  in  drawing 
of  the  fiber  and  reduced  impurities  decreased  the  attenuation  of  single-mode  fiber  signifi¬ 
cantly.  As  fiber  optic  networks  have  evolved  and  new  devices  developed,  the  advantage 
of  fiber  has  been  employed  on  an  ever  increasing  scale.  Local  area  networks  employed 
multi-mode  fiber  with  LED-based  transmitters  for  100  Mbit/  links  and  today  40  Gbit  links 
are  being  employed  in  the  core  networks  of  the  telecom  carriers.  Fiber  optic  cables  for  I/O 
connections  for  servers  have  been  growing  in  popularity  as  the  benefits  of  the  optic  com¬ 
ponents  increased  and  the  reliability  improved.  Multi-mode  fiber  is  used  extensively  for 
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short  reach  links  because  of  the  benefits  of  signal  launch  tolerance,  cleaning,  and  low  cost 
of  the  fiber  optic  transceivers. 

There  are  different  types  of  multi-mode  fiber.  The  differences  are  characterized  by  the 
over-filled  launch  bandwidth.  Due  to  its  construction,  the  bandwidth  of  the  fiber  depends 
on  the  core  diameter  and  material  effective  index  profile.  Table  12  summarizes  the  multi- 
mode  fiber  currently  available. 


Fiber 

Wavelength 

Core  Diameter 

Over  Filled 
Launch 
Bandwidth 

OM-1 

850nm 

62. Sum 

200 

1310nm 

SOO 

OM-2 

850nm 

SOum 

SOO 

1310nm 

SOO 

OM-3 

850nm 

SOum 

2000 

1310nm 

OM-3+ 

850nm 

SOum 

4700 

1310nm 

Table  12:  Fiber  types  in  production  and  their  over  filled  launch  bandwidth 


The  multi-mode  fiber  modal  properties  have  been  studied  extensively  in  the  IEEE  stan¬ 
dards  groups  to  allow  higher  data  rate  transmission  over  62.5  um  and  50  um  core  fiber. 
At  10  Gbit/s,  electrical  compensation  techniques  for  the  optical  signal  are  used  when  the 
bandwidth  of  the  fiber  is  too  low.  The  single  emitter  standard  developed  for  this  applica¬ 
tion  is  the  IEEE  802. 3aq  10  GBASE-LRM  standard.  For  server  and  SAN  applications, 
these  restrictions  are  not  typical  as  new  cable  can  be  deployed.  Multi-mode  fiber  is  cur¬ 
rently  the  fiber  of  choice  for  the  data  center. 


Figure  20:  Trend  in  data  center  fiber  purchasing 

(Courtesy  of  R.  Grzybowski,  Corning) 
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The  high  bandwidth  fiber  allows  longer  distanees  or  higher  data  rate  for  equivalent  distanee. 
From  the  trend  in  eable  type,  higher  bandwidth  fiber  is  preferred  for  new  installations 
over  smaller  bandwidth  OM-2  or  OM-1  fiber.  Fiber  length  is  also  important  to  understand 
in  terms  of  fiber  cable  interconnects.  From  the  sales  of  cable  length,  Figure  21  shows 
clearly  that  60%  of  the  cable  sold  today  is  for  distances  below  100  meters. 


Premises  Cable  Market  -  Length  Distribution  of  Cable 
Supplied  to  Corning  Customers 


Cable  Length(m) 

Length  Distribution  — Cumulative  Frequency 


CORNING  I  Sdoned  fi  Tcd«icila$iy 


Figure  21:  Distribution  of  premise  cable  length  sold  to  Corning  customers 

(Courtesy  of  R.  Grzybowski,  Corning) 


This  implies  that  100  Gbit  parallel  optic  transceiver  links  can  meet  a  100  meter  distance 
requirement  by  utilizing  OM-3  fiber. 


5.2  Optical  transceivers 

Fiber  optic  transceivers  are  utilized  in  most  network  switches  and  SAN  environments. 

For  switches,  the  preferred  transceiver  is  currently  the  pluggable  module.  For  host  bus 
adaptor  cards,  the  through-hole  version  is  preferred.  The  transceivers  typically  utilize  InP 
devices  for  long  wavelength  1310  nm  applications  and  GaAs  VCSEL  devices  for  short 
wavelength  850  nm  multi-mode  applications.  The  parallel  optical  modules  utilize  VCSEL 
arrays  for  the  SNAP- 12  and  POP-4  designs.  The  transmitter  and  receiver  devices  are  the 
core  of  the  transceiver.  The  InP  and  GaAs  material  systems  today  have  been  shown  to 
allow  modulation  from  the  p-n  junction  device  up  to  data  rates  of  40  Gbit/s. 

The  long  wavelength  devices  are  dominated  by  ridge  and  buried  heterostructure  devices. 
These  are  typically  distributed-feedback  (DFB)  or  Fabry-Perot  (FP)  laser  devices  that  are 
either  directly  modulated  or  externally  modulated.  The  cost  of  these  devices  is  volume 
and  overhead  dependent.  These  devices  are  principally  used  in  single-mode  fiber  applica¬ 
tions.  Single-mode  fiber  imposes  a  tight  alignment  tolerance  on  the  packaging  unless 
mode  expanders  or  waveguide  manipulation  are  utilized.  They  are  predominately  used  for 
long  reach  links  greater  than  300  meters. 
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The  two  principal  transceivers  module  interfaces  are  duplex  (LC/SC)  or  MPO  connector. 
Alternative  interfaces  for  transceivers  are  used  in  the  automobile  market  (plastic  optical 
fiber),  but  these  are  not  discussed  in  this  report. 

5.2.1  Single  channel  -  duplex 

There  are  multiple  types  of  fiber  connectors.  The  transceivers  that  are  shipped  in  volume 
today  are  mainly  LC  duplex  interfaces.  When  the  market  moved  from  the  1x9  footprint  to 
the  small  form  factor  footprint,  there  were  several  interfaces/connectors  initially  devel¬ 
oped  (Table  13). 


Connector 

Ferrule  size 

Fiber  spacing 

Latch 

LC 

1.25mm 

6.25mm 

RJ-top  2  latch 

MT-RJ 

2.5  X  4.4mm 

0.75mm 

RJ-top  latch 

SC-DC 

2.5mm 

0.75mm 

SC  push-pull 

VF-45 

None 

4.5mm 

RJ-top  latch 

Table  13:  Connectors  initially  developed  for  the  SFF  transceiver 


The  issues  of  repeatability,  insertion  loss,  back  reflection,  and  mechanical  stability  were 
all  investigated.  The  two  connectors  that  prevailed  were  the  LC  and  MT-RJ  connector. 
The  LC  is  most  widely  used  today  because  of  the  compatibility  with  TO-can  technology 
and  package  diameter.  The  LC  connector  mated  well  with  TO-46  and  TO-56  assembly 
techniques  for  single-mode  and  multi-mode  optical  subassemblies.  The  fiber  spacing 
provided  enough  mechanical  tolerance  to  package  two  TO  cans  side  by  side.  Other  ce¬ 
ramic  package  types  that  utilized  LC  interface  exist  and  these  were  developed  later  to 
enable  cooling,  optical  isolation,  or  high  speed  RF  feed  through. 

The  market  is  segregated  into  different  segments.  Both  Ethernet  and  SANs  are  predomi¬ 
nately  served  today  by  short  wavelength  transceivers.  Meanwhile,  D-WDM,  C-WDM, 
and  SONET/OTN  are  mainly  served  by  long  wavelength  transceivers.  The  FTTH  market 
is  different.  This  market  typically  uses  fiber  pigtailed  product  on  a  single  fiber  output/ 
input. 

The  short  wavelength  market  is  dominated  by  the  850  nm  VCSEL  device.  It  utilizes  a 
Bragg  reflector  in  the  device  structure  to  launch  the  light  through  the  top  surface  of  the 
device.  The  VCSEL  is  a  multi-mode  device  and  replaced  the  850nm  edge  emitter  tech¬ 
nology  in  Fibre  Channel  applications.  Short  wavelength  transceivers  utilize  multi-mode 
fiber  interfaces/connectors.  These  VCSEL  devices  are  widespread,  dominating  the  1  Gbit 
Ethernet  optical  transceiver  market  and  SAN  market.  The  ratio  of  short  reach  to  long 
reach  transceiver  ports  in  the  Ethernet  market  is  70:30  (850  nm:  13 10  nm)  at  1  Gbit/s. 

This  is  expected  to  reverse  at  10  GbE.  For  Ethernet,  the  SFP  and  X-type  transceivers 
dominate.  Examples  of  these  transceivers  are  shown  in  Figure  22. 
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Figure  22:  Examples  of  the  duplex  transceivers 

(Courtesy  of  W.  Jiang,  JDSU  -  01  DA  100  Gbit  Ethernet  Forum) 

At  the  higher  data  rates  of  10  Gbit/s,  additional  ICs  were  initially  incorporated  into  the 
transceivers  to  provide  signal  re-timer  or  multiplexer/de-multiplexer  functions.  The 
current  trend  in  the  industry  is  to  push  these  functions  back  into  the  SER-DES  on  the  host 
board.  This  is  expected  to  reduce  the  electrical  component  cost  within  the  transceiver. 

The  reduction  in  footprint  allows  more  ports  per  blade  to  be  incorporated,  increasing  the 
density  of  10  Gbit  duplex  transceivers  on  a  board. 

Work  is  progressing  in  the  current  Ethernet  environment  on  developing  100  Gbit  trans¬ 
ceivers.  The  exact  footprint,  power  consumption,  and  interface  are  still  not  clear.  Due  to 
cost  constraints  and  development  budgets,  the  industry  trend  is  to  utilize  current  compo¬ 
nent  technology.  This  is  an  issue  the  industry  will  need  to  address  as  demand  for  high 
speed  components  becomes  more  pressing  over  the  next  few  years. 

5.2.2  Parallel  channel 

The  parallel  fiber  architecture  for  interconnects  was  developed  initially  to  solve  the  com¬ 
puter  interconnect  problem.  A  majority  of  funding  of  the  basic  research  to  enable  these 
transceivers  was  provided  by  DARPA.  The  basic  approach  is  similar  to  the  ribbon  cable 
used  in  the  printers  and  computers  in  the  1980s.  Instead  of  parallel  electrical  connection, 
a  parallel  optical  connection  is  used.  The  transceivers  are  E-0  converters  with  parallel 
signal  inputs  (Figure  23).  The  transmission  (optical)  lines  must  be  compensated  for  skew, 
but  this  method  allows  increasingly  higher  aggregate  bandwidth  to  be  provided,  which  is 
proportional  to  the  fiber  count. 


100  Gbit  Interconnects  and  Above;  The  Need  for  Speed  -  38 
Optoelectronics  Industry  Development  Association 


Figure  23:  Parallel  optical  link  architecture  using  VCSEL  arrays 

(Courtesy  of  J.  Ojha,  Avago  Technologies) 

The  parallel  links  take  advantage  of  VCSEL  array  teehnology.  The  array  (photodiode  or 
VCSEL)  sits  on  top  of  an  alignment  substrate  or  eleetrical  IC.  The  actual  design  is  de¬ 
pendent  on  the  manufacturer.  Only  a  few  companies  have  been  successful  with  VCSEL 
array  transceiver  products.  When  these  modules  were  initially  released  during  the  optical 
bubble,  modules  could  fail  catastrophically.  The  reliability  concerns  for  parallel  trans¬ 
ceivers  stem  from  the  infant  mortality  issues  related  to  dark  line  defects  (DLD). 

In  the  parallel  module  certain  designs  require  the  VCSEL  to  sit  on  top  of  the  electrical 
circuit.  This  design  allows  thermal  transfer  from  the  IC  to  the  VCSEL  array,  increasing 
the  junction  temperature.  As  reliability  and  infant  mortality  are  accelerated  with  increas¬ 
ing  temperature  and  drive  current,  the  removal  of  the  heat  and  thermal  dissipation  re¬ 
quires  careful  design.  The  problems  of  junction  temperature  increases  and  heat  sinking  of 
parallel  modules  have  been  published  by  several  companies. 

As  the  GaAs  material  system  suffers  from  DLD,  all  VCSEL  devices  must  go  through  a 
bum-in  to  weed  out  the  weak  devices.  Typically,  the  bum-in  forces  the  DLD  to  move  into 
the  active  region  if  it  is  present.  To  overcome  this  problem,  Agilent  Technologies  (now 
Avago  Technologies)  released  several  publications  on  their  1x12  transceiver  suggesting 
that  moving  to  a  longer  wavelength  alleviated  this  problem.  By  adding  indium  into  the 
quantum  well  and  shifting  the  wavelength  to  980  nm,  they  showed  that  infant  mortality 
could  be  reduced. 
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The  advantage  the  parallel  transceiver  offers  is  that  the  aggregate  transmission  data  rate 
can  be  4  to  12  times  higher  than  the  current  duplex  transceivers.  Additionally,  a  parallel 
transceiver  can  provide  equivalent  aggregate  data  rate  to  a  duplex  transceiver  but  utilize 
lower  electrical  data  rates.  This  approach  can  alleviate  some  of  the  signal  integrity  chal¬ 
lenges  that  are  seen  at  10  Gbit/s  and  above.  Figure  24  provides  an  overview  of  a  parallel 
module. 


Multi-channel  (Parallel)  Link:  Most  Likely  Solution  in 
HPC  Environment  for  100  GbE 
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Figure  24:  High  performance  computing  parallel  transceiver  solution 

(Courtesy  of  P.  Pepeljugoski,  IBM-  OIDA  100  Gbit  Ethernet  Forum) 


The  VCSEL  bandwidth,  ESD,  and  reliability  are  related  to  the  active  region  diameter. 
Conference  publications  have  demonstrated  20  Gbit/s  modulation  in  the  laboratory.  If  the 
VCSEL  is  unable  to  achieve  higher  bandwidth,  do  we  need  to  look  at  alternative  device 
technology  for  parallel  modules?  In  the  serial  transmission  space  the  EA  modulator  or 
MZ -modulator  approaches  work  well  up  to  40  Gbit/s  but  perhaps  for  higher  data  rates  a 
different  device  will  be  required. 

Alternatives  to  the  POP-4  and  SNAP  12  transceiver  modules  have  been  developed.  The 
Quad  Small  Form  Factor  module,  which  is  based  on  the  XFP  mechanical  package,  has 
been  released  to  the  market.  It  utilizes  an  MPO  fiber  optic  connector  to  allow  parallel 
transmission  from  a  pluggable  port,  similar  to  the  current  XFP  duplex  transceiver.  The 
design  takes  advantage  of  the  XFP  surface  mount  connector,  allowing  migration  to  4x10 
Gbit  or  12x10  Gbit/s  module.  With  the  current  development  of  high  speed  surface  mount 
connectors,  these  rates  could  potentially  move  to  4x25  Gbit/s  or  12x25  Gbit/s  per  electri¬ 
cal  lane.  A  12x25  Gbit/s  parallel  module  would  provide  a  300  Gbit/s  interconnect  solu¬ 
tion. 
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For  board-to-board  optical  interconnects,  the  transceiver  packaging  will  need  to  change. 
Potential  solutions  include  co-packaging  within  the  IC  package  or  implementation  of  a 
silicon  photonics  solution.  One  of  today’s  solutions  is  to  integrate  850  nm  VCSEL  tech¬ 
nology  into  the  ball  grid  array  (EGA)  package  or  as  an  ‘add-on’  to  the  current  silicon  IC 
package.  Co-packaging  an  0-E  converter  within  the  IC  package  will  enable  on-board 
optical  interconnectivity.  Several  groups  are  using  VCSEL  technology  because  of  the  low 
drive  voltages  and  alignment  tolerance  to  multi-mode  fiber  or  optical  waveguides.  Array 
technology  is  relatively  mature  and  demonstrations  of  2-D  arrays  connecting  to  circuit 
boards  have  been  produced.  The  question  of  bum- in  of  the  array  and  packaging/align- 
ment  is  similar  to  the  parallel  module  mechanics.  This  approach  is  a  packaging  exercise 
and  very  high  precision  assembly  oriented.  To  move  to  a  lower  cost  implementation, 
wafer  scale  integration  will  need  to  occur. 

The  alternative  solution  is  currently  silicon  photonics.  This  approach  is  being  investigated 
by  several  companies,  including  Intel.  The  main  question  with  a  wafer  scale  approach  is 
generating  the  light  transmitter  or  light  source.  Several  silicon  photonic  building  blocks 
have  been  demonstrated  including  modulators,  receivers,  and  transceivers.  These  demon¬ 
strations  have  used  extrinsic  sources.  The  simplest  integration  of  the  light  source  is  cur¬ 
rently  achieved  using  standard  flip  chip  die  bonding  techniques.  To  achieve  a  wafer  scale 
source,  the  integration  into  the  silicon  fabrication  line  would  be  preferred.  One  method  is 
to  use  wafer  bonding  of  III-V  material  to  the  silicon.  Wafer  bonding  is  a  common  process 
in  the  LED  industry.  The  idea  is  simply  to  add  some  III-V  semiconductor  that  can  gener¬ 
ate  the  photons  for  the  silicon  laser  resonator.  This  topic  is  covered  in  more  detail  in  the 
OIDA  report  Silicon  Photonics:  Challenges  and  Future. 

Integrated  0-E  converters  would  enable  the  optical  PCB  industry.  The  problem  will  be 
whether  it  should  be  based  on  850  nm  VCSELs  or  1310  nm  silicon  photonics.  The  actual 
wavelength  will  need  to  be  determined  to  enable  the  right  properties  of  the  waveguides  to 
be  implemented.  1310  nm  VCSEL  technology  has  been  developed  but  is  not  in  mass 
deployment.  The  technology  remains  relatively  immature  compared  to  today’s  850  nm 
devices. 

Fundamentally,  the  conversion  to  an  optical  signal  should  be  part  of  the  silicon  IC  to  achieve 
the  lowest  cost.  The  next  section  addresses  the  state  of  play  of  optical  PCB  technology. 

5.3  On-board  waveguides 

The  basics  premise  of  the  optical  printed  circuit  board  is  to  allow  transmission  through  or 
on  the  PCB  with  an  optical  guide  that  exhibits  low  loss  and  dispersion.  There  are  several 
materials  that  can  be  utilized  to  provide  the  optical  waveguide,  including 

•  glass  fiber 

•  polymers 

•  sol  gels 

•  glass 
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The  structure  and  dimensions  of  the  waveguide  are  determined  by  the  geometry,  materi¬ 
als,  and  signal  wavelength.  As  highlighted  before,  there  are  two  distinct  approaches  being 
developed  within  the  industry: 

•  VCSEL  and  parallel  based  transmitter  technology 

•  Silicon  photonic  transmitters 

The  coupling  of  the  transmitters  to  the  optical  circuit  board  is  a  key  area  that  needs  de¬ 
velopment.  The  approach  needs  to  allow  blind  mating  with  standard  pick  and  place 
equipment.  Highly  tolerant  optical  signal  paths  need  to  be  developed  between  the 
waveguide  and  the  IC’s  optical  transmitter  and  receiver  circuitry.  Figure  25  provides  a 
conceptual  overview: 
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Figure  25:  Conceptual  overview  of  the  optical  PCB  connections 

(Courtesy  of  Peter  Van  Deale,  University  of  Ghent) 


The  current  backplanes  in  the  server  architecture  utilize  FR-4  circuit  cards  and  electrical 
connectors.  There  has  been  a  tremendous  amount  of  effort  in  optical  PCB  card  R&D. 
Optical  technology  reduces  FMI  emissions  and  works  well  with  flexible  topologies. 
Several  university  and  industrial  companies  have  been  developing  optical  waveguides.  A 
few  of  these  approaches  are  highlighted  below. 
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Method  1:  Glass  fibers  in  flex 

The  simplest  approach  is  to  use  standard  fiber.  The  fiber  can  be  placed  on  a  polymer  or 
“plastic”  backplane  and  then  routed  and  spliced  to  enable  the  I/O  required.  The  fiber  is 
sealed  within  a  polymer/plastic  for  rigidity  and  durability  (Figure  26). 


Figure  26:  Optical  fibers  in  polymer  to  form  a  flex  cable  for  waveguide  in  backplanes 

(Courtesy  of  Peter  Van  Deale,  University  of  Ghent) 

Method  2:  Polymer  waveguides 

The  concern  with  polymer  waveguides,  dating  back  to  the  first  silicon  bench  develop¬ 
ments,  is  stability.  Polymers  have  been  found  to  have  unstable  refractive  index  with 
exposure  to  moisture  environments.  Today,  polymers  have  been  developed  that  can 
provide  a  stable  refractive  index  profile  over  time.  These  polymers  can  be  fabricated  into 
waveguides  using  standard  photolithography  (Figure  27)  or  other  techniques,  including: 

•  Gray- scale  photolithography 

•  Molding 

•  Laser  ablation 

The  losses  for  the  some  of  these  waveguides  are  <0.1  dB/cm  at  850  nm  wavelength. 
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Waveguide  Fabrication 
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Figure  27:  Waveguide  fabrication  using  polymeric  materials 

(Courtesy  of  J.  V.  DeGroot,  Jr.,  Dow  Corning) 


Molding  of  the  waveguides  provides  a  road  to  mass  production.  Several  researchers  have 
published  work  where  silicon  masters  are  used  to  control  the  waveguide  mold  tolerance. 
Molding  also  offers  the  ability  to  form  different  structures.  Mirrors  can  be  incorporated 
into  the  waveguide  layout  and  produce  ultra  smooth  surfaces.  Figure  28  provides  an 
example  of  a  mirror  fabricated  with  a  polymeric  material. 


Figure  28:  Mirror  fabricated  using  a  polymeric  material 

(Courtesy  of  J.  V.  DeGroot,  Jr.,  Dow  Corning) 


100  Gbit  Interconnects  and  Above;  The  Need  for  Speed  -  44 
Optoelectronics  Industry  Development  Association 


PCB  Waveguide  connection 

The  MT  connector  is  utilized  in  fiber  optic  cable  assemblies,  which  makes  it  an  ideal 
choice  for  the  optical  PCB  and  backplane.  In  Japan,  a  government  program  has  been 
developing  optical  backplane  technology  to  America’s  Carriers  Telecommunications 
Association  (ACTA)  standards.  The  backplane  connection  utilizes  polymers  for  encasing 
the  fiber  which  are  terminated  with  an  MT  connector.  Figure  29  highlights  the  intercon¬ 
nection  scheme  under  development. 
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Figure  29:  MT  connectors  for  optical  backplane  connections 

(Courtesy  of  H.  Itoh,  AIST,  Japan) 


This  approach  enables  high  speed  optical  daughter  board  to  backplane  connection.  The 
mechanical  alignment  and  interface  is  well  understood  and  available  today.  An  alterna¬ 
tive  is  to  attach  the  MT  connector  to  the  polymer  waveguide  PCB  (Figure  30).  This  is  not 
as  elegant  as  the  glass  fiber  connection  but  it  is  a  step  forward  for  polymer-based 
waveguides. 
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MT  adapter  inserted  in  test  card:  WGs  illuminated 
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Figure  30:  MT  adapter  to  provide  an  optical  connection  from  a  PCB  card 

(Courtesy  of  R.  Grzybowski:  Corning) 


Further  research  and  development  is  required  to  push  optical  PCBs  to  the  mainstream. 
The  development  of  standards  and  optical  IC  front  ends  need  to  be  pursued  to  enable 
mass  deployment  of  this  technology. 
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6  Economics 

The  economics  for  the  implementation  of  improved  I/O  connections  is  dependent  on  the 
health  of  the  supply  chain  and  technology  investment  requirements.  It  is  important  to 
understand  both  the  needs  of  the  end  user  and  the  capabilities  of  the  current  suppliers. 

From  a  historical  look  at  the  fiber  optic  and  copper  interconnects/cable  market,  it  has 
been  assumed  that  fiber  will  increase  in  deployment  as  the  data  rate  increases.  As  new 
standards  and  data  rates  have  been  released,  however,  fiber  is  having  difficulty  displacing 
copper  solutions.  Each  time  the  optics  community  feels  that  it  will  gain  improved  share, 
the  electrical  community  provides  a  new  or  better  solution.  For  example,  at  10  Gbit 
Ethernet  it  was  believed  that  a  copper  solution  could  not  be  achieved.  Yet  a  10  Gbit 
base-T  standard  has  been  developed  and  100  Gbit  Ethernet  copper  cable  solutions  have 
been  proposed  in  the  current  IEEE  HSSG  study  group.  The  copper  connector  and  cable 
assembly  companies  are  profitable  while  the  fiber  optic  companies  remain  unprofitable  or 
are  beginning  only  now  to  break  even.  This  disparity  plays  into  the  economic  challenges 
for  high  speed  I/O.  Both  the  computing  and  data  center  markets  have  aggressive  cost 
targets.  This  can  directly  impact  the  roadmap  for  fiber  optic  interconnects,  based  on  the 
current  economic  issues  prevailing  in  the  component  industry. 

During  the  100  Gbit  Ethernet  workshop  held  in  August  2006  by  OIDA,  several  compa¬ 
nies  noted  that  there  is  a  real  need  to  increase  the  I/O  connection  speed.  This  is  becoming 
increasingly  important  in  the  high  performance  computer  (HPC)  market.  There  has  been 
fiber  optic  deployment  and  it  will  migrate  from  the  high  end  server  to  the  low  end  servers 
as  costs  decrease.  One  of  the  key  requirements  to  ensuring  successful  deployment  is  cost. 
The  estimated  cost  requirement  per  bit  of  transmitted  data  for  the  HPC  environment  is 
distance  driven  (Figure  31). 
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Challenges  for  Optical  Interconnect  at  Short  Distances 


0.01  0.1  1  10  100 


Distance  (m) 

Critical  Issues  for  Optics:  Cost,  Cost,  Cost . Rctiabtlits^ 


P<?rformance. 


IBNfResear^ 


Figure  31:  Distance  vs.  the  cost  per  bit  of  transmitted  data 

(Courtesy  of  P.  Pepeljugoski,  IBM-  OIDA  100  Gbit  Ethernet  Forum) 


The  question  of  how  to  achieve  this  with  optical  module  technology  is  challenging.  The 
cost  of  the  module,  optical  sub-components,  and  volume  play  a  large  role  in  the  econom¬ 
ics.  The  current  cost  today  of  a  1  Gbit  short  reach  (SR)  optical  module  is  nearly  equiva¬ 
lent  to  the  price  of  CatSe  cable  a  consumer  can  buy  from  Staples. .  .$20  to  $30.  For  the  10 
Gbit/s  duplex  modules,  the  price  is  an  order  of  magnitude  larger  and  currently  in  the  $300 
and  above  range.  The  comparative  cost  depends  on  the  application,  i.e.,  short  reach  mod¬ 
ule  or  long  reach  transponder. 


6. 1  Transceivers 

Duplex  transceivers  have  had  incredible  price  erosion  over  the  last  10  years.  As  new 
modules  have  been  introduced,  the  price  premium  for  each  new  introduction  has  been 
eroded.  The  price  expectation  is  dependent  on  the  market  segment.  For  example,  the 
expected  price  of  the  40  Gbit  transponder  is  about  2  to  2.5  times  that  of  the  10  G  trans¬ 
ponder  price  while  the  price  for  a  4.25  Gbit  Fibre  Channel  transceiver  is  expected  to  be 
equivalent  to  the  2.125  Gbit  Fibre  Channel  transceiver.  The  problem  with  this  expectation 
is  that  the  current  transceiver  vendors  are  struggling  to  be  profitable.  With  no  premium 
for  new  module  introduction,  there  is  very  little  money  available  to  develop  new  trans¬ 
mitter  and  receiver  technologies  to  enable  the  next  generation  of  components.  An  exam- 
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pie  of  the  price  erosion  based  on  the  cost  per  Mb/s  per  linear  inch  of  face  plate  was  pre¬ 
sented  at  the  OIDA  micropackaging  workshop  and  is  shown  below: 


Figure  32:  Price  erosion  for  Ethernet  and  Fibre  Channel 

(Courtesy  of  K.  Jackson,  EMCORE) 


The  Fibre  Channel  market  is  fortunate  that  most  of  the  investment  in  10  Gbit  and  40  Gbit 
technology  occurred  during  the  1990s,  prior  to  the  optical  bubble  bursting.  As  the  enter¬ 
prise  market  looks  at  a  projected  market  for  100  Gbit  Ethernet  of  only  100,000  units  by 
2010,  the  return  on  investment  argument  is  no  longer  valid.  As  we  move  to  these  speeds, 
the  fundamental  technology  that  was  developed  through  the  1990s  has  not  seen  any 
radical  change.  Hermitic  packages  are  preferred  and  TO-cans  remained  a  significant 
volume  runner.  The  development  of  VCSEL  technology  has  had  a  huge  impact  in  the 
storage  market,  but  long  wavelength  VCSEL  technology  has  struggled  to  break  ground. 
The  promise  of  ubiquitous  low  cost  1310  nm  VCSEL  technology  is  currently  competing 
with  low  cost  FP  and  DFB  laser  technology.  As  the  data  rate  requirements  have  in¬ 
creased,  DFB  and  FP  infrastructure,  production,  and  yields  have  improved  to  enable 
lower  manufacturing  costs.  As  the  data  rates  move  to  above  10  Gbit/s,  the  external  (lith¬ 
ium  niobate)  or  internal  (electro-absorption)  modulator  technology  is  used.  For  the  enter¬ 
prise  and  telecom  markets,  the  fiber  non-linearity  and  impairments  make  the  standard 
non-return-to-zero  (NRZ)  serial  solution  unattractive  at  40  Gbit/s  for  long  reach  applica¬ 
tions.  Additionally,  electronic  driver  technology  is  more  expensive  and  large  voltage 
swings  are  required.  At  lower  data  rates,  low  voltage  swings  and  inexpensive  IC  technol¬ 
ogy  provide  a  cost  argument  to  implement  parallel  links  over  serial  type  solutions. 
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For  current  optical  duplex  transceivers,  the  construction  and  devices  inside  the  ‘box’ 
provide  significant  cost  challenges  to  meet  a  $/Gbit  metric.  There  are  several  factors  that 
make  this  difficult.  These  include  the  module  construction,  type  of  semiconductor  device, 
and  manufacturing  location.  The  basic  components  of  a  duplex  transceiver  are  not  com¬ 
plicated,  as  shown  in  Figure  33. 


Figure  33:  Basic  components  of  a  SFP  transceiver 

(Courtesy  of  B.  Ring,  WSR-ODS) 

The  cost  of  a  transceiver  can  be  broken  down  into  several  levels.  At  the  macro  level,  the 
basic  components  are  the  electrical  and  optical  subassemblies  that  are  soldered  together 
and  put  into  the  casing.  The  subcomponents,  i.e.,  transmitter  optical  subassembly  (TOSA) 
and  receiver  optical  subassembly  (ROSA),  are  a  large  percentage  of  the  cost  in  1  Gb/s 
modules.  At  higher  data  rates,  the  electronics  are  a  larger  part  of  the  cost  breakdown.  This 
is  one  of  the  reasons  that  the  SFP-i-  module  is  being  developed. 

The  principle  sales  in  the  storage  market  are  in  VCSEL-based  products.  The  main  pack¬ 
age  used  to  produce  the  TOSA  is  the  TO-can.  The  elements  of  the  package  are  essentially 
the  following: 

•  VCSEL  die 

•  Monitor  photodiode 

•  TO-can 

•  Lens  cap 

•  ADM  assembly 

The  package  material  price  for  the  TO-can  has  been  reduced  significantly  due  to  the 
volume  market  for  CD  lasers  which  has  driven  down  manufacturing  costs.  In  volume,  the 
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package  costs  for  these  elements  are  mueh  less  than  a  U.S.  dollar.  An  example  of  a  TO- 
46  price  in  low  volume  is  provided  in  Table  14. 


Quantity  |  1000 

5000 

10000 

50000 

Description 

TO-46  $  1.00 

$  0.45 

$  0.40 

$  0.29 

Table  14:  Cost  estimates  for  a  TO-46  header 

(Private  Communication) 


The  prineiple  material  eost  driver  ean  be  the  semiconductor  device  and  its  assembly 
proeessing  eost.  The  cost  of  the  deviee  will  depend  on  the  eompany  cost  structure,  includ¬ 
ing  depreciation  and  operating  costs.  To  provide  information  relevant  to  the  debate,  Table 
15  provides  a  photodiode  cost  assumption  based  on  a  loaded  labor  rate  of  $250/hour. 


Die  Cost  model  example 

Die  Size 

SOOpm  X  300|Lim 

Assumes  1  wafer  per  batch  and  approximated 
processing  times.  Loaded  labor  rate  depends 
on  fabrication  facility  and  business  model. 
Consumables  cost  ignored. 

Useable  Wafer  Area 

40mm'^ 

Expected  Die  per  wafer  (Unyielded) 

17778 

Wafers/batch 

1 

Loaded  Labor  Rate 

$  250.00 

Grown  E PI  Wafer  Cost 

$  300.00 

Step 

Operation 

Yield 

Cum  Yield 

Time  (hrs) 

Yielded 
Material  Cost 

Yielded  Labor  &  OH 

1 

START  WAFER 

2 

DIELECTRIC  DEPOSITION 

100% 

100% 

2 

$  300.00 

$500 

9 

PHOTO:  DIFFUSION 

100% 

100% 

6 

$  300.00 

$1,500 

23 

ZINC  DIFFUSION 

90% 

90% 

5 

$  333.33 

$1,389 

30 

DIELECTRIC  DEPOSITION 

100% 

90% 

2 

$  333.33 

$500 

33 

PHOTO:  P-CONTACT 

100% 

90% 

3 

$  333.33 

$750 

47 

PHOTO:  P-METAL 

100% 

90% 

3 

$  333.33 

$750 

58 

"P"  METAL  DEPOSITION 

100% 

90% 

3 

$  333.33 

$750 

66 

WAFER  THINNING 

100% 

90% 

4 

$  333.33 

$1,000 

69 

"N"  METAL  DEPOSITION 

100% 

90% 

3 

$  333.33 

$750 

76 

PROBE  TEST 

95% 

86% 

25 

$  350.88 

$6,579 

79 

DIE  SEPARATION 

80% 

68% 

6 

$  438.60 

$1,875 

84 

WAFER  PICKING  AND  DIE  INSPECTION 

90% 

62% 

4 

$  487.33 

$1,111 

66 

$  4,210.14 

$17,454 

fotal  Cost/Wafer  = 

$  21,664.08 

Cost/die  = 

$  1.22 

Table  15:  Estimated  fabrication  cost  for  a  photodiode 

(Source:  Cost  Modei:  OtDA  tnP  Foundry  Workshop) 


The  example  in  Table  15  is  loaded  labor  rate  dependent.  Several  faetors  influence  this 
ineluding  the  proeessing  linewidth.  Assuming  a  linewidth  of  1  um  requirement  (i.e., 
active  region  width  of  a  1310  nm  semieonductor  laser),  the  fabrication  investment  would 
be  several  tens  of  millions  of  dollars  for  new  equipment.  The  facility  construction  cost 
depends  on  the  air  handling  requirements  and  elean  room  elass.  Table  16  shows  the 
approximate  cost  per  square  foot  for  a  fabrieation  faeility  vs.  air  handling  requirement. 
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Class  Limits 

ILcI  Id 

10 

100 

1000 

10000 

100000 

Air  Changes  per  hr 

600 

300-480 

150-250 

60-120 

Oct-40 

HEPA  Filter  coverage 

% 

100 

70-100 

30-60 

10-30 

5-10 

CFM  per  Sg.  Ft. 

90 

65-36 

32-18 

16-9 

8-5 

Typical  Filter  Efficiency 

99.9997 

99.997 

99.997 

99.997 

99.97 

Typical  Filter  Velocity 

60-110  FPM 

50-90  FPM 

40-90 

FPM 

25-40 

FPM 

10-30 

FPM 

Air  Flow  type 

Unidirectional 

Unidirectional 

Mixed 

Mixed 

Mixed 

Typical  Return  Air  Flow 
System 

Raised  Floor 

Low  Wall 

Low  Wall 

Low  Wall 
or  Ceiling 

Low  Wall 
or  Ceiling 

Generic  Cost  Estimate 
($  per  Sq  Ft) 

600-750 

450-650 

160-260 

60-70 

40-50 

Table  16:  Generic  cost  per  square  foot  for  a  clean  room  facility 

(Source:  OIDA  InP  Foundry  Report) 

As  the  die  size  is  reduced,  the  number  of  die  per  wafer  increases.  This  should  decrease 
die  per  wafer  cost,  but  the  loaded  labor  rates  depend  on  the  utilization  of  the  facility, 
operating  costs,  and  depreciation.  For  small  market  sizes  requiring  few  wafers  per  year, 
the  cost  of  ownership  of  the  fabrication  facility  is  questionable.  For  the  VCSEL  compo¬ 
nent,  the  additional  wafer  validation  and  bum- in  costs  are  added  to  the  die  cost.  Table  15 
highlights  that  depending  on  the  company  overhead,  the  die  can  either  be  the  most  sig¬ 
nificant  cost  in  the  module  or  have  a  comparable  cost  to  the  outside  material  purchase 
cost  of  the  TO-can. 

With  advances  in  semiconductor  packaging  and  technology,  alternative  packaging  and 
assembly  is  available  to  transceiver  manufacturers.  Principally,  they  could  leverage  the 
technology  utilized  in  LED  packaging  or  IC  manufacturing  to  reduce  costs.  LED  packag¬ 
ing  today  utilizes  lead  frames  with  high  thermal  dissipation.  To  move  to  lead  frame  pack¬ 
aging  requires  development  of  non-hermetic  VCSELs  and  laser/photodiode  devices.  A 
non-hermetic  environment  can  drastically  increase  the  infant  mortality  rate  of  active 
optical  semiconductor  devices.  This  would  necessitate  moving  to  several  of  the  tech¬ 
niques  initially  adopted  in  the  silicon  industry. 

6.2  Dilemma  with  the  economics  for  optical  component  imple¬ 
mentations 

For  optical  interconnects  to  become  more  ubiquitous  in  the  computing  arena,  there  are 
several  challenges  for  the  optical  community,  including: 
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•  Cost 

•  Performance 

•  Reliability 

•  Size 

•  Thermal  dissipation 

•  Interconnection  interface 

The  primary  targets  for  optical  component  implementation  continues  to  be  cost.  Several 
companies  from  the  computer  and  server  markets  provided  their  input.  The  targets  pro¬ 
vided  to  the  community  were: 

•  Board-to-board:  $1  Gbit 

•  Chip-to-chip:  $0.25  Gbit 

The  current  parallel  optic  transceiver  manufacturers  stated  that  perhaps  $4  Gbit  was 
achievable  today.  The  prices  are  aggressive  for  current  technology  and  packaging  ap¬ 
proaches.  In  addition,  testing  is  seen  as  a  major  obstacle.  Alternative  schemes  such  as 
those  being  pursued  using  silicon  photonics  or  integrated  III-V  OEIC  may  be  able  to  meet 
these  targets. 

The  server  architecture  today  is  not  designed  for  optical  interconnects.  This  would  need 
to  change  for  optics  to  make  progress.  The  debate  of  the  economics  of  a  copper  cable 
solution  vs.  a  fiber  optic  solution  is  multifaceted.  It  will  continue  to  progress,  but  slowly, 
as  both  economic  factors  change  and  technology  advances.  In  the  end,  if  there  is  a  will 
there  is  a  way. 
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7  Breakout  sessions 


There  were  three  breakout  discussions  held  at  the  workshop.  The  objective  was  to  debate 
the  issues  the  group  had  been  presented  during  the  day.  The  breakout  sessions  were  led 
by  three  moderators  who  provided  written  feedback  from  their  meetings  to  the  group.  The 
three  questions  debated  were: 

1 .  The  system  requirements  for  optical  and  electrical  interconnects  and  the  current 
bottlenecks:  how  will  interconnects  need  to  develop  to  address  the  increasing  speed 
requirements? 

2.  How  do  the  optical  communications  and  silicon  industries  address  the  short  reach 
interconnect  issues? 

3.  What  are  the  technology  issues  for  next  generation  active/passive  optical  and  copper 
interconnects?  Where  do  we  need  to  focus  research? 

7 A  Session  1  summary 

Session  1:  The  system  requirements  for  optical  and  electrical  interconnects  and  the 
current  bottlenecks:  how  will  interconnects  need  to  develop  to  address  the  increasing 
speed  requirements? 

1 .  There  are  a  variety  of  systems  from  small  to  large,  from  few  nodes  to  many  nodes, 
servers,  storage,  memory,  so  you  can’t  exactly  draw  a  bright  line  where  optics  or 
electronics  will  or  won’t  be  used. 

a.  Can  be  extra  value  to  optical  connection,  for  instance  orthogonal  mating  of 
boards  that  may  drive  a  decision 

2.  Different  applications 

a.  Telecom,  not  strained  today 

b.  Server  consolidation  ->  drives  bigger  pipes 

c.  High  performance  computing  target  communications  cost  10-20%  of 
system  cost 

i.  System  architecture  driven  by  tradeoffs  of  memory,  CPU  power, 
number  of  nodes,  comm  cost  (which  goes  as  n2  on  nodes) 

d.  Data  centers  -  conservative  in  technology  adoption 

3.  Today:  $3  Gb  would  be  attractive  for  optics;  $1  would  be  no-brainer  Cu  replacement 

4.  Power,  latency,  airflow/backplane,  special  value  to  optics  for  system  reasons  (e.g., 
security) 

5.  On  board  waveguides,  very  interesting  and  may  be  part  of  solution 

a.  No  infrastructure,  no  one  can  build  the  boards  for  5-7  years 
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b.  These  kinds  of  developments  may  make  possible  new  architeetures  that 
take  advantage  of  new  capabilities 


Timeframe 

2006 

2008 

2011 

2016 

Chip-chip 

$.25/G 

$0.02/G 

$0.002/G 

(~10cm)2 

ITb/s  7,8 

10Tb/s9,10,ll 

100Tb/s9 

Board-board 

$1/Gb 

$0.5/G/b 

$0.1/G 

(~lm)2 

200Gb/ch3,6 

1Tb 

5Tb 

Rack-rack 

$5/Gb 

$3/G 

$1/G 

$0.25/G 

(lO-lOOm)l 

20Gb/ch  lOOG 

*Critical  need  today 

400G 

ITb/ch 

1 .  This  market  driven  primarily  by  price 

2.  These  markets  are  driven  by  needs/performance  advantages  GIVEN  that  price  is 
“reasonable” 

3.  Current  bandwidths  range  up  to  400  G  today,  nominal  number 

4.  Price  is  the  value  at  which  optics  becomes  attractive  but  other  points  (power,  etc.) 
will  drive  ultimate  decision 

5.  Optics  may  add  additional  value  due  to  unique  capabilities  so  could  be  priced  higher 
in  some  apps 

6.  Backplane  cost  dramatically  increased  for  higher  bandwidth  by  back  drilling 

7.  Cell  processor  is  .8  Tb/s  today 

8.  Memory  wall,  application-specific  integrated  circuit  (ASIC)  pin  count  is  driving  need 
for  new  solutions 

9.  ITRS  red  brick  wall. .  .no  solutions  seem  available 

10.  Fiber  count  as  important  as  pin  count,  can’t  have  hundreds/thousands  of  fi¬ 
bers/package,  optics  can  enable  WDM,  highly  muxed  bw/fiber,  is  this  the  need  for 
SiPhot? 

1 1.  At  10  Tb  and  $0.02  G,  optics  cost  is  $200. .  .reasonable? 

7.2  Session  2  summary 

Session  2:  ‘How  do  the  optical  communications  and  silicon  industries  address  the  short 
reach  interconnect  issues? 

The  debate  centered  on  the  question  of  a  real  definition  for  short  reach.  For  the  working 
group,  we  defined  this  to  mean  inside  the  box  or  cabinet. 

For  inter-rack  connections,  the  InfiniBand  standard  has  growing  acceptance  from  ven¬ 
dors.  This  is  due  to  the  low  latency  of  the  connection.  The  working  group  discussed  many 
issues  relevant  to  the  question.  Some  of  the  key  points  raised  included: 
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1 .  We  need  predictive  failure  for  short  reach  interconnects 

2.  The  system  architecture  today  does  not  lend  itself  well  to  optical  solutions 

3.  Silicon  photonics  is  in  its  relative  infancy  compared  to  other  III-V  solutions 

a.  There  is  no  real  field  demonstration  of  hybrid  silicon  photonics  except  for 
III-V  light  emitters  bonded  to  silicon  for  the  light  source 

b.  The  failure  modes  are  not  currently  understood 

There  are  key  drivers  in  computers  that  are  different  to  traditional  optical  communica¬ 
tions  links.  The  principle  drivers  are: 

•  volume 

•  cost 

•  $Gb 

•  $  Watt 

The  current  optical  modules  that  are  used  for  inter-rack  communication  utilize  multi- 
mode  parallel  links.  These  are  expensive  but  are  forecast  to  achieve  $4  Gb.  The  target  for 
optical  modules  was  discussed  as  being  $1  Gb  for  inter-box  connections  and  lower  for 
inside  the  box.  To  reduce  the  cost  of  the  transceiver,  it  was  suggested  that  eliminating 
testing  and  providing  relaxed  specifications  would  enable  cost  reductions.  One  of  the  key 
concerns  of  the  module  manufacturers  included  connectors.  Optical  connectors  are  ex¬ 
pensive,  as  is  the  cabling.  Development  of  lower  cost  optical  connections  would  enable 
more  proliferation  of  optics  in  the  data  center. 

Alternative  solutions  that  could  reduce  this  cost  include  moving  away  from  the  traditional 
transceiver  to  on-board  optical  interconnects.  To  drive  this,  the  computer  architects  need 
to  design  for  optics.  This  will  enable  a  more  optics-friendly  environment. 

One  of  the  overriding  guidelines  that  arose  was  that  the  optical  module  or  on  board  proc¬ 
ess  needed  to  be  compatible  with  CMOS. 

7.3  Session  3  summary 

Session  3:  What  are  the  technology  issues  for  next  generation  active/passive  optical 
and  copper  interconnects?  Where  do  we  need  to  focus  research? 

This  session  focused  on  the  technology  issues  for  next  generation  requirements.  Several 
points  of  debate  were  raised  and  the  following  points  were  discussed: 

1 .  We  need  to  provide  more  consideration  for  parallel  architectures 

2.  Dealing  with  the  skin  effect  for  copper  devices  increases  the  longevity  for  intercon¬ 
nect  applications 

3.  Is  there  a  next  generation  connector  for  the  optical  fiber?  Should  the  industry  look  to 
fiber  connectivity  at  the  fiber  interface? 
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4.  There  needs  to  be  a  compatibility  of  0/E  packaging  with  IC  processing,  (e.g.,  adhe¬ 
sive  performance  with  reflow) 

To  understand  the  next  generation  requirement,  we  need  to  understand  the  next  level  of 
system  architectures.  The  key  issue  is  routing  of  the  signals  in  the  servers.  Can  we  utilize 
wavelength  division  multiplexing  (WDM),  passive  optics,  or  different  network  compo¬ 
nents?  The  group  recognized  that  technology  roadmaps  for  the  process  nodes  and  disci¬ 
plines  are  essential. 

When  we  look  at  the  data  requirements,  the  group  asked  itself,  ‘What  does  it  take  to  get 
to  Terabit  solutions?’  Is  it  WDM,  single-mode  fiber,  or  multi-mode  fiber? 

As  the  group  moved  forward,  it  asked  for  cost  effective  transmitter  and  receiver  compo¬ 
nents.  The  price  target  for  these  was  1  Gb/$.  Most  of  the  arguments  centered  on  whether 
or  not  we  can  get  to  the  low  cost  device  if  we  have  the  volume.  It  was  stated  that  this 
argument  is  similar  to  the  CD/DVD  market.  The  group  thought  that  economies  of  scale 
could  enable  this  by  taking  advantage  of  wafer  technology. 

The  copper  connector  is  familiar  for  computer  and  server  manufacturers.  The  issues  of 
signal  integrity  and  skin  effects  are  well  understood.  The  copper  companies  are  a  small 
‘group’  which  understands  the  deficiencies  and  provides  standardized  product.  To  enable 
optics  penetration  into  traditional  copper  markets  in  the  computer  industry  requires  proc¬ 
ess  standardization,  joint  efforts  between  companies,  and  clear  development  roadmaps.  It 
would  be  preferred  if  a  single  material  component  platform  could  be  used. 

Several  of  the  optical  component  vendors  suggested  that  packaging  cost  is  a  limiting 
factor.  Fiber  alignment,  testing  at  each  process  step,  increases  the  optical  device/module 
cost.  This  needs  to  be  understood  to  enable  the  cost  requirements  to  be  met.  Leveraging  a 
high  volume  application  was  recognized  as  the  route  to  achieving  this.  In  summary,  the 
group  recommended  the  following: 

1.  The  loss  needed  to  be  reduced  for  electrical  connectors.  This  would  need  different 
material  and  efforts  to  reduce  cross-talk  and  EMI. 

2.  Cost  needs  to  be  reduced  for  the  current  optical  technology  for  10  the  packaging.  This 
could  be  achieved  by  more  integration  of  the  optical  components.  The  advantages  of 
WDM  and  parallel  transmission  should  be  leveraged.  The  group  requested  that  spe¬ 
cific  roadmaps  for  components  be  generated. 

3.  New  technology  needs  to  be  implemented  for  the  next  generation  of  10.  A  single 
material  platform  would  enable  this  if  it  could  leverage  wafer  scale  methods. 
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8  Roadmap 

One  of  the  objectives  of  the  Interconnect  Forum  was  to  draw  conclusions  and  develop  a 
path  forward  for  review  within  the  industry,  i.e.,  a  roadmap.  Several  organizations  de¬ 
velop  roadmaps  in  technology  to  provide  potential  paths  forward.  The  computer  industry 
has  been  following  Moore’s  law  for  microprocessors  and  as  the  dimensions  become 
smaller,  new  effects  and  roadblocks  appear.  The  ITRS  regularly  reviews  the  current  trend 
in  IC  semiconductor  nodes  and  development  of  new  technology.  The  silicon  industry  has 
adopted  new  approaches  to  resolve  signaling  across  chips  and  to  improve  signal  integrity. 
We  have  seen  the  implementation  of  copper  and  low-k  dielectrics.  As  signaling  require¬ 
ments  increase,  3-D  stacking  and  silicon  via  technology  to  local  memory  chips  has  been 
developed.  These  methods  continuously  improve  the  performance  of  copper  intercon¬ 
nects  within  chips  and  delay  the  implementation  of  0-E  front  ends  at  the  output  pins  of 
the  IC. 

As  we  look  forward,  the  microprocessor  industry  continues  to  advance  and  enable  new 
applications.  It  is  expected  that  terabit  instructions  per  second  will  enable  a  new  host  of 
applications.  The  timeframe  for  this  is  the  next  10  to  15  years.  Figure  34  highlights  the 
development  objectives  and  potential  applications  moving  forward. 
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Figure  34:  Integration  of  memory  methodologies  being  pursued  today 

(Courtesy  of  J.  Bautista,  Intel  Corporation) 
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As  the  industry  pushes  forward  to  multi-core  processing  and  “tera-scale”  architectures  in 
the  future,  both  the  parallel  interconnection  using  copper  and  fiber  need  continued  re¬ 
search  effort. 

As  we  look  at  the  traditional  optical  component  development  and  communications  road¬ 
maps,  it  was  expected  that  SONET/OTN  would  move  to  160  Gbit/s  as  the  next  step  after 
40  Gbit/s.  Figure  35  highlights  the  trend  in  optical  communications  data  rate  develop¬ 
ment. 
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Figure  35:  Predicted  optical  bit  rate  evolution  forecast  for  2010 

(Courtesy  of  J.  Ojha,  Avago  Technologies) 


Today  we  are  seeing  several  developments  within  the  industry  in  optical  communica¬ 
tions.  The  reversal  of  OTN  vs.  Ethernet  leadership  in  data  rate  is  occurring  as  carriers  are 
aligning  their  transport  protocols  to  carrier  class  Ethernet. 

As  we  look  forward  to  the  next  generation  of  devices,  components,  and  technology,  the 
economics  of  the  industry  directly  impact  the  future  development  paths.  Optical  devel¬ 
opment  of  on-chip  optical  interconnects  is  expected  to  be  delayed  as  silicon  photonics  is 
still  immature  and  will  require  further  development. 

Looking  to  future  expectations,  the  data  rate  is  projected  to  increase  and  several  bottle¬ 
necks  currently  occurring  need  to  be  resolved.  The  2007  report  from  iNEMI  suggests  that 
optical  component  technology  is  too  immature  to  provide  optical  interconnects  even  by 
2017.  For  optical  interconnects  to  impact  board-to-board  connections,  the  implementation 
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of  the  optical  backplane  will  need  to  overcome  alternative  lower  cost  technology  options. 
It  is  expected  that  optical  backplanes  will  be  required  for  HPC  environments. 

A  few  important  “take-aways”  from  the  meeting  include: 

•  Optical  technology  needs  to  drive  to  lower  cost  and  be  compatible  with  the  silicon 
industry  process  to  enable  introduction  in  a  co-packaged  fashion. 

•  The  E-0  conversion  needs  to  be  low  power  and  low  cost. 

•  Forum  participants  expect  to  see  a  push  for  high  aggregate  bandwidth  either 
through  a  silicon  photonic  platform  or  a  III-V  semiconductor  platform. 

Figure  39  highlights  several  expectations  drawn  from  the  OIDA  meeting. 
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Figure  36:  Evolution  of  optical  components  required  for  optical  interconnects 
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9  Recommendations 


Fiber  optics  offer  clear  advantages  as  we  move  to  higher  data  rates  within  the  computer 
industry.  The  question  on  how  this  is  best  resolved  will  require  industry  collaboration  and 
greater  interaction  between  the  traditional  optical  community  and  the  silicon  semiconduc¬ 
tor  industry  association.  The  silicon  industry  will  need  to  address  the  interconnect  road¬ 
blocks  on  the  horizon  by  providing  a  change  in  the  architecture  from  electrical  to  optical. 
Several  approaches  are  being  developed  today  to  address  the  same  problem.  The  follow¬ 
ing  are  some  recommendations  resulting  from  the  many  discussions  at  the  workshop.  The 
recommendations  are  a  synopsis  of  several  of  the  debates  held  during  the  breakout  ses¬ 
sions. 

•  The  server  companies  need  to  provide  clear  guidance  on  their  fiber  optic  and  opti¬ 
cal  circuit  card  technology  requirements  if  they  wish  to  enable  adoption  within  the 
next  10  years. 

•  OIDA  should  engage  with  IPC  and  other  optic  standards  organizations  to  discuss 
how  to  standardize  optical  PCBs  and  their  optical  interface. 

•  With  both  silicon  photonics  and  VCSELs  attempting  to  address  the  optical  inter¬ 
connect  market,  standardization  on  wavelength  would  help  address  this  conun¬ 
drum. 

•  The  cost  of  an  optical  module  for  the  SAN  and  data  center  will  require  a  new 
packaging  technology  implementation.  Leveraging  the  LED  packaging  technol¬ 
ogy  should  be  considered  by  the  fiber  optic  supply  base. 

•  The  copper  connector  industry  has  defined  the  roadmap  for  next  generation  con¬ 
nectors  up  to  40  Gbit/s.  The  fiber  optic  industry  needs  to  understand  how  this  fits 
into  the  transceiver  module  requirements  and  what  additional  connector  technol¬ 
ogy  should  be  developed  in  conjunction  with  module  manufacturers. 

•  Current  packaging  of  fiber  optic  transceivers  has  migrated  to  South  East  Asia;  the 
U.S.  government  should  recognize  the  lack  of  fiber  optic  packaging  infrastructure 
in  the  U.S.  and  develop  a  program  to  improve  American  competitiveness  in  this 
area. 
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10  Summary  and  conclusions 

The  forum  held  by  OID  A  provided  valuable  insight  in  the  current  paths  being  developed 
for  optical  interconnects.  Clearly,  the  use  of  copper  interconnect  for  short  links  remains 
healthy  and  improvements  in  performance  keep  the  implementation  of  optical  connec¬ 
tions  from  becoming  mainstream. 

Several  different  technology  approaches  were  presented  that  are  being  developed.  This 
was  highlighted  at  the  Silicon  Photonics  forum  (held  the  next  day).  The  discussions  on 
architecture  highlighted  a  crucial  aspect  that  must  be  resolved  within  the  computer  indus¬ 
try  for  optics  to  advance.  The  computer  architecture  must  evolve  to  favor  optical  over 
electrical  interconnects.  Currently,  optics  is  utilized  for  high  end  rack-to-rack  connections 
using  parallel  optical  modules.  These  offer  aggregate  rates  of  30  Gbit/s  today.  The  high 
performance  computer  market  is  looking  at  100  Gbit  links  today  and  will  require  higher 
speed  links  in  the  future.  As  most  of  the  mid-range  and  low-end  servers  are  using  1  Gbit/s 
links,  the  migration  to  10  Gbit/s  links  will  occur  prior  to  the  requirement  for  100  Gbit/s. 

Some  key  metrics  were  repeated  themes  throughout  the  meeting.  Several  server  compa¬ 
nies  discussed  the  issue  of  cost  of  optical  links.  Within  the  traditional  communications 
market,  most  optical  component  vendors  are  losing  money.  The  computer  industry  pro¬ 
posed  concentrating  on  three  key  metrics:  $/Gbit,  $/W,  and  comparison  to  copper.  The 
target  of  $1 /Gbit  is  a  factor  of  4-8  lower  than  is  available  today. 

Current  optical  transceiver  technology  has  been  developed  for  the  optical  communica¬ 
tions  market.  Initially  this  was  a  high  priced,  low  volume  market.  Today  it  remains  at  low 
volume,  but  with  requirements  of  low  price  and  high  reliability.  A  paradigm  shift  in 
optical  links  will  need  to  occur  if  the  $  1/Gbit  or  below  is  to  be  achieved. 

Government  initiatives  in  Japan,  Europe,  and  the  United  States  are  pushing  for  high 
performance  and  new  technology  for  optical  computing  interconnects.  Within  the  U.S., 
DARPA  remains  committed  to  funding  new  optical  technology.  The  current  emphasis  on 
silicon  photonics  is  a  key  area  where  potential  changes  can  be  realized.  The  main  issue 
will  remain  the  development  of  a  low  cost  reliable  light  source.  III-V  photonics  offers 
this  today,  but  it  is  felt  incompatible  with  current  silicon  industry  direction.  Fitting  III-V 
into  a  CMOS  environment  provides  several  challenges  and  a  commitment  from  silicon 
companies.  A  solution  discussed  at  the  meeting  was  the  potential  development  of  a  sili¬ 
con  photonics  foundry  program,  similar  to  MOSIS.  This  might  enable  the  silicon  photon¬ 
ics  paradigm  shift  to  increase  research  and  development  in  this  area. 

There  several  different  options  today  to  produce  optical  circuit  boards.  The  level  of  so¬ 
phistication  and  maturity  in  polymer-based  waveguides  has  enabled  embedded  rigid  and 
flex  optical  circuit  cards.  The  polymer  material  can  be  produced  in  sheets  and  meet  IPC- 
650  and  GR1209  standards.  Two  critical  areas  remain  to  be  addressed:  coupling  out  of 
the  PCB  and  the  packaged  IC  optical  interface.  MT  ferrule  connectors  offer  a  solution  for 
card  edge  connection,  but  highly  tolerant  surface  coupling  remains  a  packaging  concern. 
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To  resolve  this,  the  IC  optieal  interface  needs  several  questions  answered.  There  appears 
to  be  no  common  standard  or  thinking  on  how  this  can  occur.  Several  approaches  include 
low  cost  VCSELs,  while  others  look  to  1310  nm  silicon  photonic  devices  to  provide  the 
solution. 

The  IC  industry  and  copper  interconnect  manufacturers  remain  committed  to  pushing  the 
boundary  for  electrical  interconnects.  The  roadmap  for  electrical  connectors  currently 
extends  to  40  Gbit/s.  The  introduction  of  silicon  via  interconnects  and  3-D  stacking  of 
chipsets  will  extend  the  life  of  on-chip  electrical  interconnects.  The  current  global  wire 
interconnect  problem  and  “red  brick  wall”  highlighted  by  the  ITRS  remain  to  be  resolved. 
It  is  evident  that  the  optical  technology  today  for  intra-chip  and  inter-chip  signaling 
remains  too  immature  for  introduction.  The  implementation  of  silicon  photonics  into 
traditional  communications  space,  where  they  can  add  increased  functionality  for  smaller 
package  size,  does  not  follow  the  current  trend  or  philosophy  of  switch/router  manufac¬ 
turers.  Companies,  like  Cisco  Systems,  are  pulling  out  CDR  technology  from  the  trans¬ 
ceivers  to  reduce  cost.  Marketing  of  increased  transceiver  functionality  is  a  gamble  for 
silicon  photonics  companies. 

While  the  traditional  fiber  optics  industry  is  improving  and  volumes  are  increasing,  the 
issue  remains  that  the  components  industry  is  economically  unhealthy.  The  funding  of 
new  interconnect  technology  is  continuing  but  at  a  lower  level  compared  to  10  years  ago. 
Several  device  questions  need  to  be  addressed,  including  the  limitation  of  current  low 
cost  devices  such  as  VCSELs.  Perhaps  new  module  concepts  and  packaging  are  required 
to  open  up  the  server  market  more  aggressively.  With  continuing  price  erosion  in  both  the 
storage  and  Ethernet  markets,  the  need  to  develop  new  technology  without  a  price  pre¬ 
mium  is  an  industry  issue.  As  Fibre  Channel  moves  from  2xFC  to  4xFC  to  8xFC,  how  a 
company  can  recoup  investment  costs  is  an  important  question. 

Finally,  optical  interconnects  look  to  have  a  promising  future.  As  we  move  forward,  the 
roadmap  for  this  sector  needs  to  be  further  refined.  There  remains  a  need  for  higher 
speeds  and  development  of  highly  reliable,  low  cost  interconnects.  The  challenge  to  the 
industry  is  twofold:  how  to  fund  these  requirements  and  how  to  standardize  the  output  of 
the  different  technologies  being  implemented  today. 
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Appendix  A  -  Forum  Agenda 
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February  21,  2007  -  Marriott  Santa  Clara,  Santa  Clara,  CA 


7.30  -  8.00  a.m.  Registration  and  Continental  Breakfast 

8.00-  8.10  \Ne\come  -  Michael  Lebby,  0\DA 

8.10-  8.15  Introduction  -  6/// R/ng,  WSR  Optical  Device  Solutions 


8.15- 

10.00  a.m. 

Interconnects:  Data  Centers 

8.15- 

8.35 

Hewlett  Packard  -  Terry  Morris 

8.35- 

8.55 

Zarlink  -  Stan  Swirhun 

8.55- 

9.15 

IBM  -  Petar  Pepeljugoski 

9.15- 

9.35 

Intel  -  Tom  Willis 

9.35- 

9.50 

Force  10  Networks  -  John  D’ Ambrosia 

9.50- 

10.00 

Moderated  discussion 

10.00- 

10.20 

Coffee  Break 

10.20- 

12.20  p.m. 

Interconnects:  Electrical  and  Optical 

10.20- 

10.40 

Intel  -  Jerry  Bautista 

10.40- 

11.00 

TFCG  Microsystems  Lab,  Ghent  University  -  Peter  Van  Daele 

11.00- 

11.20 

ForcelO  Networks  -  Joel  Goergen 

11.20- 

11.30 

Tyco  Electronics  -  Michael  Fogg 

11.30- 

11.50 

Agilent  Technologies  -  Greg  Le  Cheminant 

11.50- 

12.10  p.m. 

Cisco  Systems  -  Mark  Gustlin 

12.10- 

12.20 

Moderated  discussion 

12.20- 

1.30 

Lunch 

1.30- 

3.20  p.m. 

Interconnects:  Active  and  Passive  Optics 

1.30- 

1.50 

Corning  -  Richard  Grzyboski 

1.50- 

2.10 

RSoft  Design  Group  -  Zhengyu  Huang 

2.10- 

2.30 

Dow  Corning  -  Jon  DeGroot 

2.30- 

2.50 

EMCORE  -  Kenneth  Jackson 

2.50- 

3.10 

Avago  Technologies  -  Jugnu  Ojha 

3.10- 

3.20 

Moderated  discussion;  organize  breakout  sessions 
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3.20- 

6.00  p.m. 

Breakout  Discussions 

3.20- 

5.00 

Breakout  and  roadmap  discussions 

5.00- 

5.45 

Reports  from  breakout  leaders 

5.45- 

6.00 

Workshop  summary  and  concluding  remarks 

6.30- 

8.30  p.m. 

Evening  Reception 
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